Jul 13 20:12:14.206074 ip-10-0-135-149 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 13 20:12:14.206089 ip-10-0-135-149 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 13 20:12:14.206099 ip-10-0-135-149 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 13 20:12:14.206414 ip-10-0-135-149 systemd[1]: Failed to start Kubernetes Kubelet. Jul 13 20:12:24.285567 ip-10-0-135-149 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 13 20:12:24.285610 ip-10-0-135-149 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ed8563f2a0bf46dfabe09060cde9c798 -- Jul 13 20:14:26.987712 ip-10-0-135-149 systemd[1]: Starting Kubernetes Kubelet... Jul 13 20:14:27.403326 ip-10-0-135-149 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 20:14:27.403326 ip-10-0-135-149 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 20:14:27.403326 ip-10-0-135-149 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 20:14:27.403326 ip-10-0-135-149 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 20:14:27.403326 ip-10-0-135-149 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 20:14:27.404008 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.403391 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 20:14:27.407767 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407750 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:27.407767 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407765 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:27.407767 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407769 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407773 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407776 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407779 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407781 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407784 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407786 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407790 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407793 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407815 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407820 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407824 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407827 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407829 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407833 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407836 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407839 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407841 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407844 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407847 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:27.407856 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407850 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407853 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407856 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407858 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407861 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407864 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407866 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407869 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407872 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407875 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407877 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407880 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407883 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407885 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407888 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407891 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407894 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407896 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407899 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407901 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:27.408318 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407904 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407907 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407909 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407911 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407914 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407916 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407919 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407921 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407924 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407926 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407929 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407933 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407937 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407940 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407945 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407949 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407954 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407957 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407960 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:27.408802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407963 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407966 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407968 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407972 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407974 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407977 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407979 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407982 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407985 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407988 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407991 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407994 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407996 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.407999 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408002 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408004 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408006 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408009 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408011 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408014 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:27.409309 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408017 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408020 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408022 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408025 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408027 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408033 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408037 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408040 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408043 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408046 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408049 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408052 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408055 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408058 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408061 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408064 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408066 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408069 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408788 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408795 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:27.409900 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408798 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408801 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408804 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408807 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408809 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408812 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408815 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408818 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408821 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408824 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408827 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408829 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408832 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408834 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408837 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408840 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408842 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408845 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408848 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408851 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:27.410387 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408854 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408857 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408859 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408862 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408865 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408867 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408870 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408872 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408875 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408877 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408879 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408882 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408887 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408890 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408894 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408896 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408899 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408901 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408904 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:27.410897 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408906 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408909 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408911 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408914 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408917 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408919 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408922 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408924 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408927 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408929 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408932 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408935 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408938 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408940 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408943 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408946 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408948 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408951 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408954 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408956 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:27.411371 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408959 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408961 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408964 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408967 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408970 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408972 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408975 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408979 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408983 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408986 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408990 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408993 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408996 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.408999 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409002 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409004 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409007 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409010 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409012 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:27.411910 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409015 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409017 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409020 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409023 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409025 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409029 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409032 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409035 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409037 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409040 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409042 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409044 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409047 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409050 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409052 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409055 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409057 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409060 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409062 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:27.412376 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409449 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409455 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409459 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409462 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409464 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409467 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409470 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409473 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409476 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409478 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409481 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409484 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409486 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409489 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409491 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409494 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409497 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409500 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409503 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409506 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:27.412855 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409509 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409512 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409514 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409517 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409520 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409522 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409525 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409528 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409531 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409533 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409536 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409539 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409541 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409544 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409546 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409549 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409552 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409554 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409557 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:27.413327 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409559 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409561 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409564 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409567 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409569 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409572 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409575 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409577 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409580 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409583 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409585 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409588 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409591 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409593 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409596 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409600 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409604 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409607 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409610 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409613 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:27.413790 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409615 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409618 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409620 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409625 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409628 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409631 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409634 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409637 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409640 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409643 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409645 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409648 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409650 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409653 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409656 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409658 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409660 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409664 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409667 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409669 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:27.414272 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409672 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409674 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409677 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409680 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409682 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409685 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409689 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409693 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409699 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409702 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409704 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409708 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409711 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409714 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409716 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409719 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409721 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409724 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409726 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.409729 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:27.414764 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409811 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409830 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409839 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409847 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409851 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409854 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409858 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409863 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409866 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409869 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409872 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409875 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409879 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409881 2568 flags.go:64] FLAG: --cgroup-root="" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409884 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409888 2568 flags.go:64] FLAG: --client-ca-file="" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409891 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409894 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409899 2568 flags.go:64] FLAG: --cluster-domain="" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409902 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409907 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409910 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409914 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409918 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 20:14:27.415248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409921 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409924 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409928 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409931 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409934 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409937 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409940 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409943 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409947 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409950 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409953 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409957 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409960 2568 flags.go:64] FLAG: --enable-server="true" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409963 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409967 2568 flags.go:64] FLAG: --event-burst="100" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409970 2568 flags.go:64] FLAG: --event-qps="50" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409973 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409976 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409979 2568 flags.go:64] FLAG: --eviction-hard="" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409983 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409986 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409989 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409992 2568 flags.go:64] FLAG: --eviction-soft="" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409995 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.409998 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 20:14:27.415840 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410001 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410004 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410007 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410011 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410014 2568 flags.go:64] FLAG: --feature-gates="" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410017 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410020 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410023 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410027 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410030 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410033 2568 flags.go:64] FLAG: --help="false" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410036 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410039 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410042 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410045 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410048 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410051 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410054 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410057 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410060 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410063 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410066 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410069 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410072 2568 flags.go:64] FLAG: --kube-reserved="" Jul 13 20:14:27.416504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410075 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410077 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410083 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410086 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410088 2568 flags.go:64] FLAG: --lock-file="" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410091 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410094 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410097 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410102 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410105 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410109 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410112 2568 flags.go:64] FLAG: --logging-format="text" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410115 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410118 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410121 2568 flags.go:64] FLAG: --manifest-url="" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410124 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410128 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410131 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410135 2568 flags.go:64] FLAG: --max-pods="110" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410138 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410141 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410145 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410148 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410151 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410154 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 20:14:27.417093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410157 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410164 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410167 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410171 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410174 2568 flags.go:64] FLAG: --pod-cidr="" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410177 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410183 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410186 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410189 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410192 2568 flags.go:64] FLAG: --port="10250" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410196 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410199 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09ed353139f72ebbf" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410202 2568 flags.go:64] FLAG: --qos-reserved="" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410205 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410208 2568 flags.go:64] FLAG: --register-node="true" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410211 2568 flags.go:64] FLAG: --register-with-taints="" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410215 2568 flags.go:64] FLAG: --registry-burst="10" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410218 2568 flags.go:64] FLAG: --registry-qps="5" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410221 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410223 2568 flags.go:64] FLAG: --reserved-memory="" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410227 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410230 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410233 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410236 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410239 2568 flags.go:64] FLAG: --runonce="false" Jul 13 20:14:27.417780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410242 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410245 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410248 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410250 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410253 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410256 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410259 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410262 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410265 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410268 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410271 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410275 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410278 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410281 2568 flags.go:64] FLAG: --system-cgroups="" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410284 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410289 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410292 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410296 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410300 2568 flags.go:64] FLAG: --tls-min-version="" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410303 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410306 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410309 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410311 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410314 2568 flags.go:64] FLAG: --v="2" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410319 2568 flags.go:64] FLAG: --version="false" Jul 13 20:14:27.418381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410323 2568 flags.go:64] FLAG: --vmodule="" Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410327 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.410330 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410476 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410480 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410485 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410489 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410492 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410495 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410498 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410500 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410503 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410506 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410508 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410511 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410514 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410516 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410519 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410521 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410525 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:27.419337 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410528 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410530 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410533 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410536 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410539 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410541 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410544 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410547 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410549 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410552 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410557 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410560 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410562 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410565 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410567 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410570 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410572 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410575 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410577 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410580 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:27.419927 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410583 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410585 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410588 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410590 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410593 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410595 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410598 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410600 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410603 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410605 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410608 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410610 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410613 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410616 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410618 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410621 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410624 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410626 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410629 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410631 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:27.420411 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410634 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410637 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410640 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410643 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410647 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410650 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410653 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410656 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410659 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410662 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410666 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410668 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410672 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410674 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410677 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410681 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410684 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410687 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410690 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410693 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:27.420925 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410695 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410698 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410700 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410703 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410705 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410708 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410711 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410714 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410716 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410719 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410721 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410724 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410726 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410729 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410733 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410736 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410739 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410741 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410744 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:27.421473 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410746 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410749 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.410751 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.411570 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.419903 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.419919 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.420771 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.420783 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420840 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420846 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420849 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420853 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420856 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420859 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420862 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:27.421941 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420865 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420868 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420871 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420873 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420876 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420879 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420883 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420887 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420890 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420893 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420898 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420900 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420903 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420906 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420908 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420911 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420914 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420916 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420919 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420922 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:27.422313 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420925 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420928 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420932 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420935 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420938 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420941 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420944 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420946 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420949 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420952 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420954 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420957 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420960 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420962 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420965 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420967 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420970 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420973 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420975 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420979 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:27.422802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420981 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420984 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420987 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420989 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420991 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420994 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.420997 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421000 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421002 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421005 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421007 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421010 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421013 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421016 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421018 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421021 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421040 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421045 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421048 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:27.423292 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421052 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421055 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421058 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421061 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421063 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421066 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421068 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421071 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421075 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421077 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421081 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421083 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421086 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421089 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421093 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421097 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421100 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421102 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421106 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421108 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:27.423765 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421111 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421114 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421116 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421119 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421122 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421125 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421127 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421130 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421132 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421135 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421138 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421140 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421143 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.421148 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421263 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:14:27.424248 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421269 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421271 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421274 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421277 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421280 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421283 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421285 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421288 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421290 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421293 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421297 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421300 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421302 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421305 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421307 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421310 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421313 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421317 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421320 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:14:27.424629 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421323 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421326 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421329 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421332 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421335 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421338 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421342 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421346 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421349 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421352 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421355 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421358 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421361 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421363 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421366 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421368 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421371 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421374 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421378 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:14:27.425097 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421381 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421383 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421386 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421389 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421391 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421394 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421398 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421401 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421403 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421406 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421408 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421411 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421413 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421416 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421419 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421421 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421439 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421443 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421446 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421448 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:14:27.425653 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421451 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421453 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421456 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421458 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421461 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421464 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421466 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421469 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421471 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421474 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421476 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421479 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421482 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421485 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421488 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421490 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421493 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421495 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421498 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421501 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:14:27.426134 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421503 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421506 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421509 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421511 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421514 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421516 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421519 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421521 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421524 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421527 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421530 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421532 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421535 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421539 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421541 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421544 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421546 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421549 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421551 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:14:27.426651 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:27.421554 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:14:27.427101 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.421559 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 20:14:27.427101 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.421668 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 20:14:27.427101 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.424705 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 20:14:27.427101 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.426176 2568 server.go:1019] "Starting client certificate rotation" Jul 13 20:14:27.427101 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.426276 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 20:14:27.427101 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.426314 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 20:14:27.429059 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.429042 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 20:14:27.435921 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.435906 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 20:14:27.445376 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.445350 2568 log.go:25] "Validated CRI v1 runtime API" Jul 13 20:14:27.447475 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.447462 2568 log.go:25] "Validated CRI v1 image API" Jul 13 20:14:27.448202 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.448190 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 20:14:27.453321 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.453295 2568 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bcdc9c1e-0b63-4779-b4e8-852afdd184f9:/dev/nvme0n1p4 f49e8a84-8c4b-4298-8eac-d9ab606eba56:/dev/nvme0n1p3] Jul 13 20:14:27.453364 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.453321 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 20:14:27.455784 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.455764 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 20:14:27.460172 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.460069 2568 manager.go:216] Machine: {Timestamp:2026-07-13 20:14:27.458681312 +0000 UTC m=+0.351935461 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100829 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2de1c69babc7c635f9e99077717d3d SystemUUID:ec2de1c6-9bab-c7c6-35f9-e99077717d3d BootID:ed8563f2-a0bf-46df-abe0-9060cde9c798 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1b:97:e2:ab:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1b:97:e2:ab:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:ba:64:8b:42:48 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 20:14:27.460172 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.460167 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 20:14:27.460301 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.460250 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 20:14:27.463035 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.463004 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 20:14:27.463755 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.463043 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-149.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 20:14:27.464789 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.464462 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 20:14:27.464789 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.464487 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 20:14:27.464789 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.464499 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 20:14:27.465272 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.465248 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 20:14:27.466739 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.466726 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 20:14:27.470886 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.470874 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 13 20:14:27.470999 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.470990 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 20:14:27.471037 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.471019 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 13 20:14:27.471037 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.471029 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 20:14:27.471094 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.471046 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 20:14:27.471094 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.471056 2568 kubelet.go:398] "Adding apiserver pod source" Jul 13 20:14:27.471094 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.471065 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 20:14:27.471450 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.471422 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 20:14:27.471505 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.471462 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 20:14:27.474442 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.474415 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 20:14:27.476401 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.476387 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 20:14:27.476479 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.476417 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 20:14:27.476585 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.476571 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 20:14:27.476626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.476590 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 20:14:27.476626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.476598 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 20:14:27.476626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.476606 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 20:14:27.476626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.476614 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 20:14:27.476626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.476619 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 20:14:27.477180 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.477170 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 20:14:27.477212 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.477182 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 20:14:27.477212 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.477191 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 20:14:27.477212 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.477197 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 20:14:27.477212 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.477205 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 20:14:27.477212 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.477213 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 20:14:27.478658 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.478643 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 20:14:27.478707 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.478662 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 20:14:27.482097 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.482082 2568 server.go:1268] "Started kubelet" Jul 13 20:14:27.482186 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.482157 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 20:14:27.482249 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.482160 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 20:14:27.482249 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.482235 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 20:14:27.482591 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.482569 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-67sdj" Jul 13 20:14:27.483002 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.482956 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 20:14:27.483091 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.482995 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-149.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 20:14:27.483091 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.483026 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-149.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 20:14:27.483064 ip-10-0-135-149 systemd[1]: Started Kubernetes Kubelet. Jul 13 20:14:27.484958 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.484944 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 13 20:14:27.486420 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.486399 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 20:14:27.489721 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.489700 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 20:14:27.490491 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.489472 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-149.ec2.internal.18c1f29942eb3919 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-149.ec2.internal,UID:ip-10-0-135-149.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-149.ec2.internal,},FirstTimestamp:2026-07-13 20:14:27.482056985 +0000 UTC m=+0.375311180,LastTimestamp:2026-07-13 20:14:27.482056985 +0000 UTC m=+0.375311180,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-149.ec2.internal,}" Jul 13 20:14:27.490781 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.490767 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-67sdj" Jul 13 20:14:27.492233 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.492218 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 20:14:27.492301 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.492219 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 20:14:27.492364 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.492337 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:27.493066 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493049 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 20:14:27.493230 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493219 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 20:14:27.493346 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493332 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 20:14:27.493439 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493406 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 20:14:27.493439 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493422 2568 factory.go:55] Registering systemd factory Jul 13 20:14:27.493537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493447 2568 factory.go:223] Registration of the systemd container factory successfully Jul 13 20:14:27.493537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493506 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 13 20:14:27.493537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493517 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 13 20:14:27.493871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493859 2568 factory.go:153] Registering CRI-O factory Jul 13 20:14:27.493913 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493873 2568 factory.go:223] Registration of the crio container factory successfully Jul 13 20:14:27.493913 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493891 2568 factory.go:103] Registering Raw factory Jul 13 20:14:27.493913 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.493900 2568 manager.go:1196] Started watching for new ooms in manager Jul 13 20:14:27.494452 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.494421 2568 manager.go:349] Starting recovery of all containers Jul 13 20:14:27.496037 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.496016 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-149.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 20:14:27.496174 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.496156 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 20:14:27.506925 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.506757 2568 manager.go:354] Recovery completed Jul 13 20:14:27.511251 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.511237 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:27.513024 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.513006 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:27.513089 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.513039 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:27.513089 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.513054 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:27.513643 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.513627 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 20:14:27.513643 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.513640 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 20:14:27.513778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.513657 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 13 20:14:27.513778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.513630 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:27.514392 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.514377 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:27.514473 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.514417 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:27.514556 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.514544 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:27.515002 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.514989 2568 policy_none.go:49] "None policy: Start" Jul 13 20:14:27.515047 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.515005 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 20:14:27.515047 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.515015 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 20:14:27.516231 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.516219 2568 policy_none.go:47] "Start" Jul 13 20:14:27.549081 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.549061 2568 manager.go:337] "Starting Device Plugin manager" Jul 13 20:14:27.549179 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.549097 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 20:14:27.549179 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.549108 2568 server.go:87] "Starting device plugin registration server" Jul 13 20:14:27.549408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.549387 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 20:14:27.549504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.549406 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 20:14:27.549602 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.549579 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 20:14:27.549826 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.549712 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 20:14:27.549826 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.549724 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 20:14:27.550143 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.550120 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 20:14:27.550213 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.550173 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:27.648251 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.648194 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 20:14:27.649513 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.649497 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 20:14:27.649601 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.649501 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:27.649601 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.649542 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 20:14:27.649601 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.649572 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 20:14:27.649730 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.649614 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 20:14:27.650487 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.650469 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:27.650590 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.650494 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:27.650590 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.650506 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:27.650590 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.650536 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.652463 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.652448 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:14:27.661312 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.661258 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.661312 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.661284 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-149.ec2.internal\": node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:27.749992 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.749943 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-149.ec2.internal"] Jul 13 20:14:27.750146 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.750050 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:27.751404 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.751382 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:27.751553 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.751418 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:27.751553 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.751453 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:27.753119 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.753103 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:27.753253 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.753237 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.753311 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.753269 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:27.754049 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.754033 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:27.754135 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.754061 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:27.754135 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.754071 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:27.754135 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.754032 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:27.754246 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.754137 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:27.754246 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.754149 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:27.755541 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.755524 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.755623 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.755549 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:14:27.757590 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.757576 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:14:27.757647 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.757600 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:14:27.757647 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.757609 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:14:27.759358 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.759337 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:27.782248 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.782220 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-149.ec2.internal\" not found" node="ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.785816 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.785796 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-149.ec2.internal\" not found" node="ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.795168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.795147 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd7b253ebc7ef0c8197aec5203560fb7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal\" (UID: \"bd7b253ebc7ef0c8197aec5203560fb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.795241 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.795171 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd7b253ebc7ef0c8197aec5203560fb7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal\" (UID: \"bd7b253ebc7ef0c8197aec5203560fb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.795241 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.795188 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66cb28dc9c7115d14af06ad19fd0f8b8-config\") pod \"kube-apiserver-proxy-ip-10-0-135-149.ec2.internal\" (UID: \"66cb28dc9c7115d14af06ad19fd0f8b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.860295 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.860240 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:27.895877 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.895848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd7b253ebc7ef0c8197aec5203560fb7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal\" (UID: \"bd7b253ebc7ef0c8197aec5203560fb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.896023 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.895881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd7b253ebc7ef0c8197aec5203560fb7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal\" (UID: \"bd7b253ebc7ef0c8197aec5203560fb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.896023 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.895907 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66cb28dc9c7115d14af06ad19fd0f8b8-config\") pod \"kube-apiserver-proxy-ip-10-0-135-149.ec2.internal\" (UID: \"66cb28dc9c7115d14af06ad19fd0f8b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.896023 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.895940 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66cb28dc9c7115d14af06ad19fd0f8b8-config\") pod \"kube-apiserver-proxy-ip-10-0-135-149.ec2.internal\" (UID: \"66cb28dc9c7115d14af06ad19fd0f8b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.896023 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.895940 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd7b253ebc7ef0c8197aec5203560fb7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal\" (UID: \"bd7b253ebc7ef0c8197aec5203560fb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.896023 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:27.895944 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd7b253ebc7ef0c8197aec5203560fb7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal\" (UID: \"bd7b253ebc7ef0c8197aec5203560fb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" Jul 13 20:14:27.961357 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:27.961287 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:28.061755 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:28.061723 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:28.084908 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:28.084889 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" Jul 13 20:14:28.089174 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:28.088997 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-149.ec2.internal" Jul 13 20:14:28.162590 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:28.162553 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:28.263168 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:28.263104 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:28.363620 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:28.363586 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:28.425256 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:28.425232 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 20:14:28.425762 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:28.425400 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 20:14:28.464586 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:28.464549 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:28.496918 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:28.496876 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 20:09:29 +0000 UTC" deadline="2027-12-17 21:43:51.470410688 +0000 UTC" Jul 13 20:14:28.496918 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:28.496909 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12529h29m22.973504413s" Jul 13 20:14:28.520569 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:28.520514 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:14:28.565568 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:28.565537 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:28.666158 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:28.666125 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:28.670709 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:28.670690 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:14:30.513252 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:30.513225 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:30.613719 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:30.613687 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:30.650429 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:30.650377 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7b253ebc7ef0c8197aec5203560fb7.slice/crio-d435420749169bad749c6f8debb036749523bb9111a46c25891ba38d9c9bbb5b WatchSource:0}: Error finding container d435420749169bad749c6f8debb036749523bb9111a46c25891ba38d9c9bbb5b: Status 404 returned error can't find the container with id d435420749169bad749c6f8debb036749523bb9111a46c25891ba38d9c9bbb5b Jul 13 20:14:30.654661 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:30.654642 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:14:30.714059 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:30.714004 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-149.ec2.internal\" not found" Jul 13 20:14:30.783280 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:30.783256 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:14:30.840488 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:30.840390 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" Jul 13 20:14:30.851791 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:30.851768 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 20:14:30.852888 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:30.852875 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-149.ec2.internal" Jul 13 20:14:30.862686 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:30.862659 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 20:14:31.219699 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.219598 2568 apiserver.go:52] "Watching apiserver" Jul 13 20:14:31.228128 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.228106 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 20:14:31.229547 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.229524 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-9xn9w","openshift-network-diagnostics/network-check-target-bmv9n","openshift-network-operator/iptables-alerter-flnnl","openshift-ovn-kubernetes/ovnkube-node-nblwk","kube-system/global-pull-secret-syncer-4hjb2","kube-system/konnectivity-agent-plm24","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9","openshift-dns/node-resolver-l8rnl","openshift-image-registry/node-ca-fdrjt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal","openshift-multus/multus-additional-cni-plugins-9z8ck","openshift-multus/network-metrics-daemon-jtdht","kube-system/kube-apiserver-proxy-ip-10-0-135-149.ec2.internal","openshift-cluster-node-tuning-operator/tuned-gzlmf"] Jul 13 20:14:31.231271 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.231248 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.232865 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.232844 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.234007 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.233842 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 20:14:31.234007 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.233883 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 20:14:31.234007 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.233903 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 20:14:31.234007 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.233912 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-prtlx\"" Jul 13 20:14:31.234312 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.234207 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-flnnl" Jul 13 20:14:31.235023 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.235007 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 20:14:31.235121 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.235074 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6nrlx\"" Jul 13 20:14:31.235121 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.235011 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 20:14:31.235222 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.235181 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 20:14:31.235272 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.235261 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 20:14:31.235357 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.235328 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 20:14:31.235944 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.235727 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.236424 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.236393 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 20:14:31.236536 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.236519 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:14:31.236602 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.236522 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 20:14:31.237143 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.237123 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-99rpl\"" Jul 13 20:14:31.237750 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.237731 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vpjls\"" Jul 13 20:14:31.238363 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.238155 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 20:14:31.238363 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.238200 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 20:14:31.238363 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.238147 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 20:14:31.238574 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.238461 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:31.238574 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.238479 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 20:14:31.238574 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.238520 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:31.238734 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.238580 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 20:14:31.238734 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.238619 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 20:14:31.240217 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.240027 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:31.240217 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.240043 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 20:14:31.241360 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.241344 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fdrjt" Jul 13 20:14:31.242131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.242114 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 20:14:31.242221 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.242117 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n9v4r\"" Jul 13 20:14:31.242221 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.242135 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 20:14:31.243264 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.243026 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.243685 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.243535 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 20:14:31.243685 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.243555 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 20:14:31.243685 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.243564 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2hcqn\"" Jul 13 20:14:31.243685 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.243569 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 20:14:31.244673 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.244645 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:31.244763 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.244709 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:31.245181 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.245162 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:14:31.245764 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.245720 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5tk4c\"" Jul 13 20:14:31.246872 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.246255 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 20:14:31.246872 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.246679 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l8rnl" Jul 13 20:14:31.250012 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.249321 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.250012 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.249804 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w77b6\"" Jul 13 20:14:31.250012 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.249806 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 20:14:31.250012 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.249896 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 20:14:31.251101 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251082 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:31.251192 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.251148 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:31.251369 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d49f5e7f-dc27-40e2-a16b-a738f7a23ce3-agent-certs\") pod \"konnectivity-agent-plm24\" (UID: \"d49f5e7f-dc27-40e2-a16b-a738f7a23ce3\") " pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:31.251519 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251382 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/116a96e7-2d64-44a3-8e91-6681027a23ed-serviceca\") pod \"node-ca-fdrjt\" (UID: \"116a96e7-2d64-44a3-8e91-6681027a23ed\") " pod="openshift-image-registry/node-ca-fdrjt" Jul 13 20:14:31.251519 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251434 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.251643 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251572 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/09d72f4e-4a0a-45c3-be16-f3c675544067-kubelet-config\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:31.251643 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251607 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-run-netns\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.251643 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251631 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-run-ovn\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.251790 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251654 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.251790 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251733 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 20:14:31.251879 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251679 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-env-overrides\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.251879 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251815 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d49f5e7f-dc27-40e2-a16b-a738f7a23ce3-konnectivity-ca\") pod \"konnectivity-agent-plm24\" (UID: \"d49f5e7f-dc27-40e2-a16b-a738f7a23ce3\") " pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:31.251879 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251868 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-etc-selinux\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.252025 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251896 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gsw6\" (UniqueName: \"kubernetes.io/projected/913e2ffc-7181-4b33-b196-5f295fadccfa-kube-api-access-2gsw6\") pod \"iptables-alerter-flnnl\" (UID: \"913e2ffc-7181-4b33-b196-5f295fadccfa\") " pod="openshift-network-operator/iptables-alerter-flnnl" Jul 13 20:14:31.252025 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251919 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-run-systemd\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252025 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251941 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-ovnkube-config\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252025 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251964 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-registration-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.252025 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.251986 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-sys-fs\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.252025 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-cnibin\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.252302 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252029 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-os-release\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.252302 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252072 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-kubelet\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252302 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252111 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-systemd-units\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252302 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252143 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-slash\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252302 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252170 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-run-openvswitch\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252302 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252182 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-68595\"" Jul 13 20:14:31.252302 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252195 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x7dp\" (UniqueName: \"kubernetes.io/projected/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-kube-api-access-2x7dp\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252302 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252226 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/09d72f4e-4a0a-45c3-be16-f3c675544067-dbus\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:31.252302 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252260 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:31.252302 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252283 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-etc-openvswitch\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252305 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-log-socket\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252347 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-socket-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.252797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252378 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np8c7\" (UniqueName: \"kubernetes.io/projected/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-kube-api-access-np8c7\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.252797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252587 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-cni-binary-copy\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.252797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252622 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.252797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252674 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/913e2ffc-7181-4b33-b196-5f295fadccfa-iptables-alerter-script\") pod \"iptables-alerter-flnnl\" (UID: \"913e2ffc-7181-4b33-b196-5f295fadccfa\") " pod="openshift-network-operator/iptables-alerter-flnnl" Jul 13 20:14:31.252797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252721 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-var-lib-openvswitch\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252744 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-node-log\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252770 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.252797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252793 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-cni-bin\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.253206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252823 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-device-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.253206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252866 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.253206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252903 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/913e2ffc-7181-4b33-b196-5f295fadccfa-host-slash\") pod \"iptables-alerter-flnnl\" (UID: \"913e2ffc-7181-4b33-b196-5f295fadccfa\") " pod="openshift-network-operator/iptables-alerter-flnnl" Jul 13 20:14:31.253206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252927 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-cni-netd\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.253206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.252982 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-ovnkube-script-lib\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.253206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.253006 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/116a96e7-2d64-44a3-8e91-6681027a23ed-host\") pod \"node-ca-fdrjt\" (UID: \"116a96e7-2d64-44a3-8e91-6681027a23ed\") " pod="openshift-image-registry/node-ca-fdrjt" Jul 13 20:14:31.253206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.253028 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rphtv\" (UniqueName: \"kubernetes.io/projected/116a96e7-2d64-44a3-8e91-6681027a23ed-kube-api-access-rphtv\") pod \"node-ca-fdrjt\" (UID: \"116a96e7-2d64-44a3-8e91-6681027a23ed\") " pod="openshift-image-registry/node-ca-fdrjt" Jul 13 20:14:31.253206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.253052 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-ovn-node-metrics-cert\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.253206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.253076 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.253206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.253106 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-system-cni-dir\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.253206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.253132 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8t5w\" (UniqueName: \"kubernetes.io/projected/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-kube-api-access-w8t5w\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.253700 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.253680 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 20:14:31.277175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.277153 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-d6zdp" Jul 13 20:14:31.283980 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.283956 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-d6zdp" Jul 13 20:14:31.305072 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.305050 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:14:31.342024 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.341999 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 20:14:31.354209 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354176 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-run-systemd\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.354375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-ovnkube-config\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.354375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354241 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-registration-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.354375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354260 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-run-systemd\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.354375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354265 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-sys-fs\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.354375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354310 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-cnibin\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.354375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354317 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-sys-fs\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.354375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354352 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-cnibin\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.354375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354345 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-kubelet\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354386 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-kubelet\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354387 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-slash\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354437 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-run-openvswitch\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354459 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-registration-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-systemd\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354501 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-slash\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354523 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-run-openvswitch\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354545 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-var-lib-kubelet\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354567 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354583 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-socket-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354604 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-os-release\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354689 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-var-lib-openvswitch\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354715 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-socket-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354734 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-node-log\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.354751 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:31.354763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354768 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-device-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354767 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-var-lib-openvswitch\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354823 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-node-log\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354795 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-sysconfig\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.354841 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret podName:09d72f4e-4a0a-45c3-be16-f3c675544067 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:31.854804238 +0000 UTC m=+3.001071398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret") pod "global-pull-secret-syncer-4hjb2" (UID: "09d72f4e-4a0a-45c3-be16-f3c675544067") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354854 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-device-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354895 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-system-cni-dir\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354909 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-ovnkube-config\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354922 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/913e2ffc-7181-4b33-b196-5f295fadccfa-host-slash\") pod \"iptables-alerter-flnnl\" (UID: \"913e2ffc-7181-4b33-b196-5f295fadccfa\") " pod="openshift-network-operator/iptables-alerter-flnnl" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354968 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/913e2ffc-7181-4b33-b196-5f295fadccfa-host-slash\") pod \"iptables-alerter-flnnl\" (UID: \"913e2ffc-7181-4b33-b196-5f295fadccfa\") " pod="openshift-network-operator/iptables-alerter-flnnl" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354970 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-cni-netd\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.354995 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-cni-netd\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355021 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-ovnkube-script-lib\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355060 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/116a96e7-2d64-44a3-8e91-6681027a23ed-host\") pod \"node-ca-fdrjt\" (UID: \"116a96e7-2d64-44a3-8e91-6681027a23ed\") " pod="openshift-image-registry/node-ca-fdrjt" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355096 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rphtv\" (UniqueName: \"kubernetes.io/projected/116a96e7-2d64-44a3-8e91-6681027a23ed-kube-api-access-rphtv\") pod \"node-ca-fdrjt\" (UID: \"116a96e7-2d64-44a3-8e91-6681027a23ed\") " pod="openshift-image-registry/node-ca-fdrjt" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355127 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/868f0604-458b-4ba7-a928-5cc701834300-hosts-file\") pod \"node-resolver-l8rnl\" (UID: \"868f0604-458b-4ba7-a928-5cc701834300\") " pod="openshift-dns/node-resolver-l8rnl" Jul 13 20:14:31.355474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355173 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-modprobe-d\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355230 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-tuned\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355276 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355330 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8t5w\" (UniqueName: \"kubernetes.io/projected/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-kube-api-access-w8t5w\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355356 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03d83af4-37e2-41d8-8b53-62a26fa8bf09-tmp\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355382 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngrct\" (UniqueName: \"kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct\") pod \"network-check-target-bmv9n\" (UID: \"94d176bf-61f1-47be-8342-638623aaffba\") " pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355408 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-conf-dir\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355458 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d49f5e7f-dc27-40e2-a16b-a738f7a23ce3-agent-certs\") pod \"konnectivity-agent-plm24\" (UID: \"d49f5e7f-dc27-40e2-a16b-a738f7a23ce3\") " pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355482 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-sys\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355515 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-socket-dir-parent\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355549 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-run-netns\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355570 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-env-overrides\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355611 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-etc-selinux\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355641 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355664 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6thx9\" (UniqueName: \"kubernetes.io/projected/868f0604-458b-4ba7-a928-5cc701834300-kube-api-access-6thx9\") pod \"node-resolver-l8rnl\" (UID: \"868f0604-458b-4ba7-a928-5cc701834300\") " pod="openshift-dns/node-resolver-l8rnl" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355695 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-cnibin\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355716 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-run-k8s-cni-cncf-io\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.356168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-var-lib-cni-bin\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355840 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-var-lib-kubelet\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.355873 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/116a96e7-2d64-44a3-8e91-6681027a23ed-host\") pod \"node-ca-fdrjt\" (UID: \"116a96e7-2d64-44a3-8e91-6681027a23ed\") " pod="openshift-image-registry/node-ca-fdrjt" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356016 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tp45\" (UniqueName: \"kubernetes.io/projected/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-kube-api-access-8tp45\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356046 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-ovnkube-script-lib\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356073 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-systemd-units\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356158 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-etc-selinux\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356211 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x7dp\" (UniqueName: \"kubernetes.io/projected/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-kube-api-access-2x7dp\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356217 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356226 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-env-overrides\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356248 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-kubernetes\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356277 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwvmn\" (UniqueName: \"kubernetes.io/projected/03d83af4-37e2-41d8-8b53-62a26fa8bf09-kube-api-access-kwvmn\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356328 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-run-netns\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356511 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-systemd-units\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356550 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-cni-dir\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356589 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356603 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-daemon-config\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.357261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356637 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/09d72f4e-4a0a-45c3-be16-f3c675544067-dbus\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356721 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-etc-openvswitch\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356743 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-log-socket\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356748 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/09d72f4e-4a0a-45c3-be16-f3c675544067-dbus\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356762 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np8c7\" (UniqueName: \"kubernetes.io/projected/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-kube-api-access-np8c7\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356783 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-cni-binary-copy\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356785 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356792 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-etc-openvswitch\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356803 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/868f0604-458b-4ba7-a928-5cc701834300-tmp-dir\") pod \"node-resolver-l8rnl\" (UID: \"868f0604-458b-4ba7-a928-5cc701834300\") " pod="openshift-dns/node-resolver-l8rnl" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.356837 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/913e2ffc-7181-4b33-b196-5f295fadccfa-iptables-alerter-script\") pod \"iptables-alerter-flnnl\" (UID: \"913e2ffc-7181-4b33-b196-5f295fadccfa\") " pod="openshift-network-operator/iptables-alerter-flnnl" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357022 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357063 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-cni-bin\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357090 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357137 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-cni-bin\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357176 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-host\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357200 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-cni-binary-copy\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357223 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-run-netns\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.358073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-hostroot\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357278 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-run-multus-certs\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357304 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-etc-kubernetes\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-ovn-node-metrics-cert\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357357 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-system-cni-dir\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357382 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-run\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357383 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-cni-binary-copy\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357459 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-system-cni-dir\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357461 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-lib-modules\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-var-lib-cni-multus\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357521 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp26h\" (UniqueName: \"kubernetes.io/projected/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-kube-api-access-zp26h\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357549 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/116a96e7-2d64-44a3-8e91-6681027a23ed-serviceca\") pod \"node-ca-fdrjt\" (UID: \"116a96e7-2d64-44a3-8e91-6681027a23ed\") " pod="openshift-image-registry/node-ca-fdrjt" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357577 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357597 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-log-socket\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357604 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-sysctl-d\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357543 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/913e2ffc-7181-4b33-b196-5f295fadccfa-iptables-alerter-script\") pod \"iptables-alerter-flnnl\" (UID: \"913e2ffc-7181-4b33-b196-5f295fadccfa\") " pod="openshift-network-operator/iptables-alerter-flnnl" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357642 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-sysctl-conf\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.358871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357671 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357711 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/09d72f4e-4a0a-45c3-be16-f3c675544067-kubelet-config\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357735 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-run-ovn\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357762 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357788 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d49f5e7f-dc27-40e2-a16b-a738f7a23ce3-konnectivity-ca\") pod \"konnectivity-agent-plm24\" (UID: \"d49f5e7f-dc27-40e2-a16b-a738f7a23ce3\") " pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357814 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-os-release\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gsw6\" (UniqueName: \"kubernetes.io/projected/913e2ffc-7181-4b33-b196-5f295fadccfa-kube-api-access-2gsw6\") pod \"iptables-alerter-flnnl\" (UID: \"913e2ffc-7181-4b33-b196-5f295fadccfa\") " pod="openshift-network-operator/iptables-alerter-flnnl" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357857 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.357972 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.358094 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/116a96e7-2d64-44a3-8e91-6681027a23ed-serviceca\") pod \"node-ca-fdrjt\" (UID: \"116a96e7-2d64-44a3-8e91-6681027a23ed\") " pod="openshift-image-registry/node-ca-fdrjt" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.358155 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-run-ovn\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.358154 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.358220 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-os-release\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.358233 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/09d72f4e-4a0a-45c3-be16-f3c675544067-kubelet-config\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.358287 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.358928 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d49f5e7f-dc27-40e2-a16b-a738f7a23ce3-konnectivity-ca\") pod \"konnectivity-agent-plm24\" (UID: \"d49f5e7f-dc27-40e2-a16b-a738f7a23ce3\") " pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:31.359684 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:31.359451 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66cb28dc9c7115d14af06ad19fd0f8b8.slice/crio-8968551b7925f508269d8d6f502d9553f9ec1851a2180710229f34aeafc25f7c WatchSource:0}: Error finding container 8968551b7925f508269d8d6f502d9553f9ec1851a2180710229f34aeafc25f7c: Status 404 returned error can't find the container with id 8968551b7925f508269d8d6f502d9553f9ec1851a2180710229f34aeafc25f7c Jul 13 20:14:31.360474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.360397 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d49f5e7f-dc27-40e2-a16b-a738f7a23ce3-agent-certs\") pod \"konnectivity-agent-plm24\" (UID: \"d49f5e7f-dc27-40e2-a16b-a738f7a23ce3\") " pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:31.360608 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.360583 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-ovn-node-metrics-cert\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.367573 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.367532 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8t5w\" (UniqueName: \"kubernetes.io/projected/d53e230c-cc46-44e1-a8f3-6cc9766fbf55-kube-api-access-w8t5w\") pod \"multus-additional-cni-plugins-9z8ck\" (UID: \"d53e230c-cc46-44e1-a8f3-6cc9766fbf55\") " pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.368438 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.368365 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x7dp\" (UniqueName: \"kubernetes.io/projected/94fa8c97-cdf1-4b58-9fdb-5a4de75009ae-kube-api-access-2x7dp\") pod \"ovnkube-node-nblwk\" (UID: \"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.370054 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.370032 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gsw6\" (UniqueName: \"kubernetes.io/projected/913e2ffc-7181-4b33-b196-5f295fadccfa-kube-api-access-2gsw6\") pod \"iptables-alerter-flnnl\" (UID: \"913e2ffc-7181-4b33-b196-5f295fadccfa\") " pod="openshift-network-operator/iptables-alerter-flnnl" Jul 13 20:14:31.370157 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.370062 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np8c7\" (UniqueName: \"kubernetes.io/projected/8396ce0a-7dc4-4e1f-aea0-32851b0c2402-kube-api-access-np8c7\") pod \"aws-ebs-csi-driver-node-wzqv9\" (UID: \"8396ce0a-7dc4-4e1f-aea0-32851b0c2402\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.370257 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.370235 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rphtv\" (UniqueName: \"kubernetes.io/projected/116a96e7-2d64-44a3-8e91-6681027a23ed-kube-api-access-rphtv\") pod \"node-ca-fdrjt\" (UID: \"116a96e7-2d64-44a3-8e91-6681027a23ed\") " pod="openshift-image-registry/node-ca-fdrjt" Jul 13 20:14:31.401626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.401577 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-149.ec2.internal" event={"ID":"66cb28dc9c7115d14af06ad19fd0f8b8","Type":"ContainerStarted","Data":"8968551b7925f508269d8d6f502d9553f9ec1851a2180710229f34aeafc25f7c"} Jul 13 20:14:31.402779 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.402756 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" event={"ID":"bd7b253ebc7ef0c8197aec5203560fb7","Type":"ContainerStarted","Data":"d435420749169bad749c6f8debb036749523bb9111a46c25891ba38d9c9bbb5b"} Jul 13 20:14:31.458631 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458593 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-os-release\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.458631 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458639 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-sysconfig\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.458631 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458663 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-system-cni-dir\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458703 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/868f0604-458b-4ba7-a928-5cc701834300-hosts-file\") pod \"node-resolver-l8rnl\" (UID: \"868f0604-458b-4ba7-a928-5cc701834300\") " pod="openshift-dns/node-resolver-l8rnl" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458727 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-modprobe-d\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458738 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-sysconfig\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458743 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-os-release\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458756 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-tuned\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458782 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03d83af4-37e2-41d8-8b53-62a26fa8bf09-tmp\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458798 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-system-cni-dir\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngrct\" (UniqueName: \"kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct\") pod \"network-check-target-bmv9n\" (UID: \"94d176bf-61f1-47be-8342-638623aaffba\") " pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-conf-dir\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458841 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/868f0604-458b-4ba7-a928-5cc701834300-hosts-file\") pod \"node-resolver-l8rnl\" (UID: \"868f0604-458b-4ba7-a928-5cc701834300\") " pod="openshift-dns/node-resolver-l8rnl" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458864 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-modprobe-d\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458893 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-sys\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458923 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-socket-dir-parent\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thx9\" (UniqueName: \"kubernetes.io/projected/868f0604-458b-4ba7-a928-5cc701834300-kube-api-access-6thx9\") pod \"node-resolver-l8rnl\" (UID: \"868f0604-458b-4ba7-a928-5cc701834300\") " pod="openshift-dns/node-resolver-l8rnl" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458975 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-cnibin\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458993 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-sys\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.458989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.458998 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-run-k8s-cni-cncf-io\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459043 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-run-k8s-cni-cncf-io\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459095 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-socket-dir-parent\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459185 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-conf-dir\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459320 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-cnibin\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459349 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-var-lib-cni-bin\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459386 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-var-lib-kubelet\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459445 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tp45\" (UniqueName: \"kubernetes.io/projected/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-kube-api-access-8tp45\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459448 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-var-lib-kubelet\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459466 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-var-lib-cni-bin\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459474 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-kubernetes\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459499 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwvmn\" (UniqueName: \"kubernetes.io/projected/03d83af4-37e2-41d8-8b53-62a26fa8bf09-kube-api-access-kwvmn\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459522 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-cni-dir\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459546 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-daemon-config\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459549 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-kubernetes\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459577 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/868f0604-458b-4ba7-a928-5cc701834300-tmp-dir\") pod \"node-resolver-l8rnl\" (UID: \"868f0604-458b-4ba7-a928-5cc701834300\") " pod="openshift-dns/node-resolver-l8rnl" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459604 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-host\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459618 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-cni-dir\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460016 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459627 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-cni-binary-copy\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459650 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-run-netns\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459671 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-hostroot\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459694 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-run-multus-certs\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459719 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-etc-kubernetes\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459751 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-run\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459756 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-run-netns\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459777 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-lib-modules\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-var-lib-cni-multus\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459816 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-host\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459856 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-var-lib-cni-multus\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459844 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-host-run-multus-certs\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459879 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-run\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459904 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-etc-kubernetes\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.459995 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-lib-modules\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460047 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-hostroot\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460076 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp26h\" (UniqueName: \"kubernetes.io/projected/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-kube-api-access-zp26h\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460103 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-sysctl-d\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.460916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460146 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-sysctl-conf\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460161 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/868f0604-458b-4ba7-a928-5cc701834300-tmp-dir\") pod \"node-resolver-l8rnl\" (UID: \"868f0604-458b-4ba7-a928-5cc701834300\") " pod="openshift-dns/node-resolver-l8rnl" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460171 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460217 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-multus-daemon-config\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460225 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-systemd\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460250 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-sysctl-d\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460270 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-var-lib-kubelet\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460275 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-systemd\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460291 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-cni-binary-copy\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460366 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-var-lib-kubelet\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.460396 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-sysctl-conf\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.460407 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.460525 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs podName:eaff4f62-5b33-40cf-ab33-ea75668f2ebb nodeName:}" failed. No retries permitted until 2026-07-13 20:14:31.960507193 +0000 UTC m=+3.106774348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs") pod "network-metrics-daemon-jtdht" (UID: "eaff4f62-5b33-40cf-ab33-ea75668f2ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.461522 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03d83af4-37e2-41d8-8b53-62a26fa8bf09-etc-tuned\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.461704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.461567 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03d83af4-37e2-41d8-8b53-62a26fa8bf09-tmp\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.467640 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.467619 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:31.467763 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.467646 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:31.467763 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.467661 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ngrct for pod openshift-network-diagnostics/network-check-target-bmv9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:31.467763 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.467719 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct podName:94d176bf-61f1-47be-8342-638623aaffba nodeName:}" failed. No retries permitted until 2026-07-13 20:14:31.967699858 +0000 UTC m=+3.113967000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ngrct" (UniqueName: "kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct") pod "network-check-target-bmv9n" (UID: "94d176bf-61f1-47be-8342-638623aaffba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:31.468525 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.468493 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp26h\" (UniqueName: \"kubernetes.io/projected/d1a320fc-c21b-4a93-a1c7-d984bc40b4ee-kube-api-access-zp26h\") pod \"multus-9xn9w\" (UID: \"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee\") " pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.469365 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.469326 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6thx9\" (UniqueName: \"kubernetes.io/projected/868f0604-458b-4ba7-a928-5cc701834300-kube-api-access-6thx9\") pod \"node-resolver-l8rnl\" (UID: \"868f0604-458b-4ba7-a928-5cc701834300\") " pod="openshift-dns/node-resolver-l8rnl" Jul 13 20:14:31.469896 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.469853 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwvmn\" (UniqueName: \"kubernetes.io/projected/03d83af4-37e2-41d8-8b53-62a26fa8bf09-kube-api-access-kwvmn\") pod \"tuned-gzlmf\" (UID: \"03d83af4-37e2-41d8-8b53-62a26fa8bf09\") " pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.470029 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.470006 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tp45\" (UniqueName: \"kubernetes.io/projected/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-kube-api-access-8tp45\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:31.543367 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.543332 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" Jul 13 20:14:31.552613 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.552569 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9z8ck" Jul 13 20:14:31.561814 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.561792 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-flnnl" Jul 13 20:14:31.567787 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.567766 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:31.574788 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.574768 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:31.582519 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.582498 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fdrjt" Jul 13 20:14:31.589495 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.589478 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" Jul 13 20:14:31.595384 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.595363 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l8rnl" Jul 13 20:14:31.600436 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.600401 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9xn9w" Jul 13 20:14:31.843451 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:31.843403 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd53e230c_cc46_44e1_a8f3_6cc9766fbf55.slice/crio-6005e1089632920ff1dd3cb4fc28860a67a2462b3fa2cad8cb04ce9e89c8161a WatchSource:0}: Error finding container 6005e1089632920ff1dd3cb4fc28860a67a2462b3fa2cad8cb04ce9e89c8161a: Status 404 returned error can't find the container with id 6005e1089632920ff1dd3cb4fc28860a67a2462b3fa2cad8cb04ce9e89c8161a Jul 13 20:14:31.860891 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:31.860833 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94fa8c97_cdf1_4b58_9fdb_5a4de75009ae.slice/crio-632d94bc72d971cde7a915f15991a794fe6fa783d48471e94d1b8dd4ccec629b WatchSource:0}: Error finding container 632d94bc72d971cde7a915f15991a794fe6fa783d48471e94d1b8dd4ccec629b: Status 404 returned error can't find the container with id 632d94bc72d971cde7a915f15991a794fe6fa783d48471e94d1b8dd4ccec629b Jul 13 20:14:31.862842 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.862814 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:31.862958 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.862927 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:31.863028 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.863012 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret podName:09d72f4e-4a0a-45c3-be16-f3c675544067 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:32.862990092 +0000 UTC m=+4.009257256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret") pod "global-pull-secret-syncer-4hjb2" (UID: "09d72f4e-4a0a-45c3-be16-f3c675544067") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:31.863654 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:31.863632 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod913e2ffc_7181_4b33_b196_5f295fadccfa.slice/crio-150065f4802a60b9a36c06586d3b72151fbc13235bd6226e2a16cc880ad0e6fd WatchSource:0}: Error finding container 150065f4802a60b9a36c06586d3b72151fbc13235bd6226e2a16cc880ad0e6fd: Status 404 returned error can't find the container with id 150065f4802a60b9a36c06586d3b72151fbc13235bd6226e2a16cc880ad0e6fd Jul 13 20:14:31.864463 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:31.864406 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod868f0604_458b_4ba7_a928_5cc701834300.slice/crio-61e6295f1c1c1e0864c417b6a02822c7980844dd744047a84cb533ae396b8e49 WatchSource:0}: Error finding container 61e6295f1c1c1e0864c417b6a02822c7980844dd744047a84cb533ae396b8e49: Status 404 returned error can't find the container with id 61e6295f1c1c1e0864c417b6a02822c7980844dd744047a84cb533ae396b8e49 Jul 13 20:14:31.865356 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:31.865328 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d83af4_37e2_41d8_8b53_62a26fa8bf09.slice/crio-a6b140d742b090914d662b5097b14008ef32479e2a8ff40d3b8c00ae8dad2f61 WatchSource:0}: Error finding container a6b140d742b090914d662b5097b14008ef32479e2a8ff40d3b8c00ae8dad2f61: Status 404 returned error can't find the container with id a6b140d742b090914d662b5097b14008ef32479e2a8ff40d3b8c00ae8dad2f61 Jul 13 20:14:31.866674 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:31.866518 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1a320fc_c21b_4a93_a1c7_d984bc40b4ee.slice/crio-d54eccc1573920e28325d2fc9ce3f074e8be026eb95639f4be070fcd7a519913 WatchSource:0}: Error finding container d54eccc1573920e28325d2fc9ce3f074e8be026eb95639f4be070fcd7a519913: Status 404 returned error can't find the container with id d54eccc1573920e28325d2fc9ce3f074e8be026eb95639f4be070fcd7a519913 Jul 13 20:14:31.868171 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:31.867677 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod116a96e7_2d64_44a3_8e91_6681027a23ed.slice/crio-0c825df16b8b7f81179ad6da064e5b1c51a99b19063031823a4015b1f2153bee WatchSource:0}: Error finding container 0c825df16b8b7f81179ad6da064e5b1c51a99b19063031823a4015b1f2153bee: Status 404 returned error can't find the container with id 0c825df16b8b7f81179ad6da064e5b1c51a99b19063031823a4015b1f2153bee Jul 13 20:14:31.868922 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:31.868489 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8396ce0a_7dc4_4e1f_aea0_32851b0c2402.slice/crio-9aa82c5ed661904a028063d4da0456e2fb91a3a36f6ceaff3c4f9c7c0921cfa9 WatchSource:0}: Error finding container 9aa82c5ed661904a028063d4da0456e2fb91a3a36f6ceaff3c4f9c7c0921cfa9: Status 404 returned error can't find the container with id 9aa82c5ed661904a028063d4da0456e2fb91a3a36f6ceaff3c4f9c7c0921cfa9 Jul 13 20:14:31.869988 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:14:31.869667 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49f5e7f_dc27_40e2_a16b_a738f7a23ce3.slice/crio-59b4b83efb9b65c9b626edd87b65daa41a109894369f30e71a8d52f20e662cc0 WatchSource:0}: Error finding container 59b4b83efb9b65c9b626edd87b65daa41a109894369f30e71a8d52f20e662cc0: Status 404 returned error can't find the container with id 59b4b83efb9b65c9b626edd87b65daa41a109894369f30e71a8d52f20e662cc0 Jul 13 20:14:31.964186 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:31.964143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:31.964316 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.964297 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:31.964373 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:31.964362 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs podName:eaff4f62-5b33-40cf-ab33-ea75668f2ebb nodeName:}" failed. No retries permitted until 2026-07-13 20:14:32.964347266 +0000 UTC m=+4.110614407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs") pod "network-metrics-daemon-jtdht" (UID: "eaff4f62-5b33-40cf-ab33-ea75668f2ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:32.065357 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.064801 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngrct\" (UniqueName: \"kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct\") pod \"network-check-target-bmv9n\" (UID: \"94d176bf-61f1-47be-8342-638623aaffba\") " pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:32.065357 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:32.064962 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:32.065357 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:32.064987 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:32.065357 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:32.064999 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ngrct for pod openshift-network-diagnostics/network-check-target-bmv9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:32.065357 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:32.065056 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct podName:94d176bf-61f1-47be-8342-638623aaffba nodeName:}" failed. No retries permitted until 2026-07-13 20:14:33.065037573 +0000 UTC m=+4.211304712 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngrct" (UniqueName: "kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct") pod "network-check-target-bmv9n" (UID: "94d176bf-61f1-47be-8342-638623aaffba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:32.286746 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.286468 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 20:09:31 +0000 UTC" deadline="2028-04-02 01:42:57.059985026 +0000 UTC" Jul 13 20:14:32.286746 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.286743 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15077h28m24.773249193s" Jul 13 20:14:32.411516 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.411224 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" event={"ID":"8396ce0a-7dc4-4e1f-aea0-32851b0c2402","Type":"ContainerStarted","Data":"9aa82c5ed661904a028063d4da0456e2fb91a3a36f6ceaff3c4f9c7c0921cfa9"} Jul 13 20:14:32.422606 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.422529 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fdrjt" event={"ID":"116a96e7-2d64-44a3-8e91-6681027a23ed","Type":"ContainerStarted","Data":"0c825df16b8b7f81179ad6da064e5b1c51a99b19063031823a4015b1f2153bee"} Jul 13 20:14:32.450405 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.450322 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" event={"ID":"03d83af4-37e2-41d8-8b53-62a26fa8bf09","Type":"ContainerStarted","Data":"a6b140d742b090914d662b5097b14008ef32479e2a8ff40d3b8c00ae8dad2f61"} Jul 13 20:14:32.467250 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.466705 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l8rnl" event={"ID":"868f0604-458b-4ba7-a928-5cc701834300","Type":"ContainerStarted","Data":"61e6295f1c1c1e0864c417b6a02822c7980844dd744047a84cb533ae396b8e49"} Jul 13 20:14:32.477259 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.477186 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-flnnl" event={"ID":"913e2ffc-7181-4b33-b196-5f295fadccfa","Type":"ContainerStarted","Data":"150065f4802a60b9a36c06586d3b72151fbc13235bd6226e2a16cc880ad0e6fd"} Jul 13 20:14:32.488710 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.488677 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" event={"ID":"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae","Type":"ContainerStarted","Data":"632d94bc72d971cde7a915f15991a794fe6fa783d48471e94d1b8dd4ccec629b"} Jul 13 20:14:32.501251 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.501216 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z8ck" event={"ID":"d53e230c-cc46-44e1-a8f3-6cc9766fbf55","Type":"ContainerStarted","Data":"6005e1089632920ff1dd3cb4fc28860a67a2462b3fa2cad8cb04ce9e89c8161a"} Jul 13 20:14:32.507252 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.507222 2568 generic.go:356] "Generic (PLEG): container finished" podID="bd7b253ebc7ef0c8197aec5203560fb7" containerID="e179c32a9d263fcc8de69a4fa5d7e3d756c3958e9da3a434176b3964fa4693f2" exitCode=0 Jul 13 20:14:32.507378 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.507302 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" event={"ID":"bd7b253ebc7ef0c8197aec5203560fb7","Type":"ContainerDied","Data":"e179c32a9d263fcc8de69a4fa5d7e3d756c3958e9da3a434176b3964fa4693f2"} Jul 13 20:14:32.513639 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.513559 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-plm24" event={"ID":"d49f5e7f-dc27-40e2-a16b-a738f7a23ce3","Type":"ContainerStarted","Data":"59b4b83efb9b65c9b626edd87b65daa41a109894369f30e71a8d52f20e662cc0"} Jul 13 20:14:32.525129 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.524067 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9xn9w" event={"ID":"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee","Type":"ContainerStarted","Data":"d54eccc1573920e28325d2fc9ce3f074e8be026eb95639f4be070fcd7a519913"} Jul 13 20:14:32.872265 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.871678 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:32.872265 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:32.871840 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:32.872265 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:32.871912 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret podName:09d72f4e-4a0a-45c3-be16-f3c675544067 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:34.871894403 +0000 UTC m=+6.018161545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret") pod "global-pull-secret-syncer-4hjb2" (UID: "09d72f4e-4a0a-45c3-be16-f3c675544067") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:32.973531 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:32.973141 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:32.973531 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:32.973362 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:32.973531 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:32.973456 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs podName:eaff4f62-5b33-40cf-ab33-ea75668f2ebb nodeName:}" failed. No retries permitted until 2026-07-13 20:14:34.973408139 +0000 UTC m=+6.119675280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs") pod "network-metrics-daemon-jtdht" (UID: "eaff4f62-5b33-40cf-ab33-ea75668f2ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:33.074018 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:33.073978 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngrct\" (UniqueName: \"kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct\") pod \"network-check-target-bmv9n\" (UID: \"94d176bf-61f1-47be-8342-638623aaffba\") " pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:33.074190 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:33.074178 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:33.074251 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:33.074205 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:33.074251 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:33.074220 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ngrct for pod openshift-network-diagnostics/network-check-target-bmv9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:33.074368 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:33.074280 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct podName:94d176bf-61f1-47be-8342-638623aaffba nodeName:}" failed. No retries permitted until 2026-07-13 20:14:35.07426236 +0000 UTC m=+6.220529526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngrct" (UniqueName: "kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct") pod "network-check-target-bmv9n" (UID: "94d176bf-61f1-47be-8342-638623aaffba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:33.287250 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:33.287141 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 20:09:31 +0000 UTC" deadline="2028-02-03 13:41:30.21663208 +0000 UTC" Jul 13 20:14:33.287250 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:33.287199 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13673h26m56.929437117s" Jul 13 20:14:33.400705 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:33.399860 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:33.400705 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:33.400013 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:33.400705 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:33.400426 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:33.400705 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:33.400531 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:33.400705 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:33.400593 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:33.400705 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:33.400662 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:34.893360 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:34.893320 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:34.893829 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:34.893520 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:34.893829 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:34.893584 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret podName:09d72f4e-4a0a-45c3-be16-f3c675544067 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:38.893564988 +0000 UTC m=+10.039832142 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret") pod "global-pull-secret-syncer-4hjb2" (UID: "09d72f4e-4a0a-45c3-be16-f3c675544067") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:34.994640 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:34.994560 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:34.994805 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:34.994749 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:34.994879 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:34.994814 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs podName:eaff4f62-5b33-40cf-ab33-ea75668f2ebb nodeName:}" failed. No retries permitted until 2026-07-13 20:14:38.994794896 +0000 UTC m=+10.141062038 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs") pod "network-metrics-daemon-jtdht" (UID: "eaff4f62-5b33-40cf-ab33-ea75668f2ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:35.095482 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:35.095371 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngrct\" (UniqueName: \"kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct\") pod \"network-check-target-bmv9n\" (UID: \"94d176bf-61f1-47be-8342-638623aaffba\") " pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:35.095679 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:35.095571 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:35.095679 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:35.095591 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:35.095679 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:35.095605 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ngrct for pod openshift-network-diagnostics/network-check-target-bmv9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:35.095679 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:35.095671 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct podName:94d176bf-61f1-47be-8342-638623aaffba nodeName:}" failed. No retries permitted until 2026-07-13 20:14:39.095652859 +0000 UTC m=+10.241920013 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngrct" (UniqueName: "kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct") pod "network-check-target-bmv9n" (UID: "94d176bf-61f1-47be-8342-638623aaffba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:35.399544 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:35.399486 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:35.399709 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:35.399606 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:35.399709 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:35.399671 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:35.399850 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:35.399758 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:35.399850 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:35.399806 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:35.399938 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:35.399876 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:37.396961 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:37.396924 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:37.397452 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:37.397063 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:37.397516 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:37.397482 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:37.397610 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:37.397573 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:37.397723 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:37.397708 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:37.397815 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:37.397797 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:38.931050 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:38.931003 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:38.931490 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:38.931157 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:38.931490 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:38.931234 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret podName:09d72f4e-4a0a-45c3-be16-f3c675544067 nodeName:}" failed. No retries permitted until 2026-07-13 20:14:46.931213516 +0000 UTC m=+18.077480658 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret") pod "global-pull-secret-syncer-4hjb2" (UID: "09d72f4e-4a0a-45c3-be16-f3c675544067") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:39.032967 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:39.032394 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:39.032967 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:39.032555 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:39.032967 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:39.032620 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs podName:eaff4f62-5b33-40cf-ab33-ea75668f2ebb nodeName:}" failed. No retries permitted until 2026-07-13 20:14:47.03260056 +0000 UTC m=+18.178867715 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs") pod "network-metrics-daemon-jtdht" (UID: "eaff4f62-5b33-40cf-ab33-ea75668f2ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:39.133149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:39.133090 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngrct\" (UniqueName: \"kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct\") pod \"network-check-target-bmv9n\" (UID: \"94d176bf-61f1-47be-8342-638623aaffba\") " pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:39.133344 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:39.133268 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:39.133344 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:39.133296 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:39.133344 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:39.133310 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ngrct for pod openshift-network-diagnostics/network-check-target-bmv9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:39.133596 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:39.133374 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct podName:94d176bf-61f1-47be-8342-638623aaffba nodeName:}" failed. No retries permitted until 2026-07-13 20:14:47.133355273 +0000 UTC m=+18.279622417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngrct" (UniqueName: "kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct") pod "network-check-target-bmv9n" (UID: "94d176bf-61f1-47be-8342-638623aaffba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:39.398797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:39.398048 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:39.398797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:39.398180 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:39.398797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:39.398199 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:39.398797 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:39.398291 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:39.398797 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:39.398667 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:39.398797 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:39.398761 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:41.397257 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:41.397217 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:41.397257 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:41.397250 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:41.397816 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:41.397217 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:41.397816 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:41.397352 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:41.397816 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:41.397457 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:41.397816 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:41.397567 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:43.397125 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:43.397090 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:43.397125 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:43.397123 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:43.397680 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:43.397089 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:43.397680 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:43.397221 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:43.397680 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:43.397291 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:43.397680 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:43.397359 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:45.397116 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:45.397075 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:45.397540 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:45.397075 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:45.397540 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:45.397207 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:45.397540 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:45.397204 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:45.397540 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:45.397316 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:45.397540 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:45.397430 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:46.993348 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:46.993311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:46.993930 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:46.993498 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:46.993930 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:46.993582 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret podName:09d72f4e-4a0a-45c3-be16-f3c675544067 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:02.993560246 +0000 UTC m=+34.139827405 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret") pod "global-pull-secret-syncer-4hjb2" (UID: "09d72f4e-4a0a-45c3-be16-f3c675544067") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:14:47.093875 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:47.093841 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:47.094044 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:47.093986 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:47.094044 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:47.094042 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs podName:eaff4f62-5b33-40cf-ab33-ea75668f2ebb nodeName:}" failed. No retries permitted until 2026-07-13 20:15:03.094025764 +0000 UTC m=+34.240292902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs") pod "network-metrics-daemon-jtdht" (UID: "eaff4f62-5b33-40cf-ab33-ea75668f2ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:14:47.194789 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:47.194760 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngrct\" (UniqueName: \"kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct\") pod \"network-check-target-bmv9n\" (UID: \"94d176bf-61f1-47be-8342-638623aaffba\") " pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:47.194952 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:47.194898 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:14:47.194952 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:47.194920 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:14:47.194952 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:47.194934 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ngrct for pod openshift-network-diagnostics/network-check-target-bmv9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:47.195073 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:47.194995 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct podName:94d176bf-61f1-47be-8342-638623aaffba nodeName:}" failed. No retries permitted until 2026-07-13 20:15:03.194977942 +0000 UTC m=+34.341245087 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngrct" (UniqueName: "kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct") pod "network-check-target-bmv9n" (UID: "94d176bf-61f1-47be-8342-638623aaffba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:14:47.396930 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:47.396893 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:47.397099 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:47.396891 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:47.397099 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:47.397016 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:47.397099 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:47.397016 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:47.397099 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:47.397084 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:47.397284 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:47.397165 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:49.397985 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:49.397948 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:49.398472 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:49.398073 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:49.398472 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:49.398091 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:49.398472 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:49.398206 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:49.398472 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:49.398237 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:49.398472 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:49.398283 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:51.398086 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.397829 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:51.398579 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.397829 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:51.398579 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:51.398193 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:51.398579 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.397880 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:51.398579 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:51.398245 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:51.398579 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:51.398312 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:51.570003 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.569768 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" event={"ID":"8396ce0a-7dc4-4e1f-aea0-32851b0c2402","Type":"ContainerStarted","Data":"0a056fb2aa2d8ce2f47e5cdbf07eaf9da64b7319b0067d2726158a493833ff2b"} Jul 13 20:14:51.571654 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.571625 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fdrjt" event={"ID":"116a96e7-2d64-44a3-8e91-6681027a23ed","Type":"ContainerStarted","Data":"3a145bb62100e29cd8c1b7986f810ac9fb47a39dc530011a4d5db93a874309e2"} Jul 13 20:14:51.573308 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.573275 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" event={"ID":"03d83af4-37e2-41d8-8b53-62a26fa8bf09","Type":"ContainerStarted","Data":"ff5c7a7091a6e8f33783629b7408d37d870bc1f9daa877ad23d8318238fbad44"} Jul 13 20:14:51.575234 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.575206 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l8rnl" event={"ID":"868f0604-458b-4ba7-a928-5cc701834300","Type":"ContainerStarted","Data":"d9463c4e627090c995c7271c6d0afa456cfd17b8e40f8ed1163cc9c3dfb5887c"} Jul 13 20:14:51.578488 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.578471 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:14:51.578956 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.578933 2568 generic.go:356] "Generic (PLEG): container finished" podID="94fa8c97-cdf1-4b58-9fdb-5a4de75009ae" containerID="2c2613e316ec8f3060e76900913d4ed5b82e17a49ec001a4576e3b37c9fe85ef" exitCode=1 Jul 13 20:14:51.579041 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.579005 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" event={"ID":"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae","Type":"ContainerStarted","Data":"88a5ce6890e5404636412af49156cacdf1fb99b90d2306aae0473bdacbe2a21e"} Jul 13 20:14:51.579112 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.579097 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" event={"ID":"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae","Type":"ContainerStarted","Data":"1b1be969f6ad77a380112b79cd47a08281237983ecdd356b8d8c64913daaecd5"} Jul 13 20:14:51.579168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.579154 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" event={"ID":"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae","Type":"ContainerStarted","Data":"0369f6214b1d00a0a9c25493eb85e2ec88e9b480a79c93959f90c5e8f501f675"} Jul 13 20:14:51.579228 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.579205 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" event={"ID":"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae","Type":"ContainerStarted","Data":"18e8566eb75dab0e433c80bba3415269a626c36039e2dd35aec9cc4ff6682925"} Jul 13 20:14:51.579294 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.579279 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" event={"ID":"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae","Type":"ContainerDied","Data":"2c2613e316ec8f3060e76900913d4ed5b82e17a49ec001a4576e3b37c9fe85ef"} Jul 13 20:14:51.579342 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.579331 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" event={"ID":"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae","Type":"ContainerStarted","Data":"28f6c24de3c297ddff3c45f9a2e5dab57150d7afc665c2ff20d7e53e7910d9f8"} Jul 13 20:14:51.580980 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.580910 2568 generic.go:356] "Generic (PLEG): container finished" podID="d53e230c-cc46-44e1-a8f3-6cc9766fbf55" containerID="8631aa678e71251246858265e8fbf4b38ad39dd265e4b09c930385fd55443c52" exitCode=0 Jul 13 20:14:51.581064 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.580990 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z8ck" event={"ID":"d53e230c-cc46-44e1-a8f3-6cc9766fbf55","Type":"ContainerDied","Data":"8631aa678e71251246858265e8fbf4b38ad39dd265e4b09c930385fd55443c52"} Jul 13 20:14:51.583502 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.583478 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" event={"ID":"bd7b253ebc7ef0c8197aec5203560fb7","Type":"ContainerStarted","Data":"33632b99777b286520d48f4ba4718396ef07f0f0c134dfedcb514a919406eec5"} Jul 13 20:14:51.585312 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.585291 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-plm24" event={"ID":"d49f5e7f-dc27-40e2-a16b-a738f7a23ce3","Type":"ContainerStarted","Data":"daa35e314d9eb5b1f3287111f27d3cd95e249df45051bf944c6cb3f280ac469d"} Jul 13 20:14:51.587005 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.586982 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9xn9w" event={"ID":"d1a320fc-c21b-4a93-a1c7-d984bc40b4ee","Type":"ContainerStarted","Data":"a4e7f6188f71732812c27bc576173b94158f7f904487dbc0af4008e8d5ed5669"} Jul 13 20:14:51.588916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.588894 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-149.ec2.internal" event={"ID":"66cb28dc9c7115d14af06ad19fd0f8b8","Type":"ContainerStarted","Data":"7073b9e3b377b4472a1bc87da62812b6b1b6f66c167fd74c6ec4ca0201fab311"} Jul 13 20:14:51.594873 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.594828 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fdrjt" podStartSLOduration=3.616517527 podStartE2EDuration="22.594815927s" totalImagesPullingTime="18.9782984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:51.594168213 +0000 UTC m=+22.740435375" watchObservedRunningTime="2026-07-13 20:14:51.594815927 +0000 UTC m=+22.741083087" Jul 13 20:14:51.619404 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.619310 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-plm24" podStartSLOduration=3.61439892 podStartE2EDuration="22.619300434s" totalImagesPullingTime="19.004901514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:51.618947214 +0000 UTC m=+22.765214376" watchObservedRunningTime="2026-07-13 20:14:51.619300434 +0000 UTC m=+22.765567595" Jul 13 20:14:51.670914 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.670856 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gzlmf" podStartSLOduration=3.655144955 podStartE2EDuration="22.670834521s" totalImagesPullingTime="19.015689566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:51.669755135 +0000 UTC m=+22.816022296" watchObservedRunningTime="2026-07-13 20:14:51.670834521 +0000 UTC m=+22.817101683" Jul 13 20:14:51.713929 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.713872 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l8rnl" podStartSLOduration=3.665120349 podStartE2EDuration="22.713855756s" totalImagesPullingTime="19.048735407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:51.691141854 +0000 UTC m=+22.837409015" watchObservedRunningTime="2026-07-13 20:14:51.713855756 +0000 UTC m=+22.860122917" Jul 13 20:14:51.714200 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.714180 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9xn9w" podStartSLOduration=3.63239185 podStartE2EDuration="22.714173322s" totalImagesPullingTime="19.081781472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:51.713535021 +0000 UTC m=+22.859802183" watchObservedRunningTime="2026-07-13 20:14:51.714173322 +0000 UTC m=+22.860440482" Jul 13 20:14:51.732474 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.732430 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-149.ec2.internal" podStartSLOduration=21.73239812 podStartE2EDuration="21.73239812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:51.7322533 +0000 UTC m=+22.878520461" watchObservedRunningTime="2026-07-13 20:14:51.73239812 +0000 UTC m=+22.878665280" Jul 13 20:14:51.752933 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:51.752894 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-149.ec2.internal" podStartSLOduration=21.752880975 podStartE2EDuration="21.752880975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:51.752570385 +0000 UTC m=+22.898837547" watchObservedRunningTime="2026-07-13 20:14:51.752880975 +0000 UTC m=+22.899148132" Jul 13 20:14:52.288174 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:52.288147 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 20:14:52.307393 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:52.307294 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T20:14:52.288170664Z","UUID":"08df00f6-0074-459e-aff1-d39eab486478","Handler":null,"Name":"","Endpoint":""} Jul 13 20:14:52.308594 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:52.308576 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 20:14:52.308670 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:52.308623 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 20:14:52.592823 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:52.592792 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-flnnl" event={"ID":"913e2ffc-7181-4b33-b196-5f295fadccfa","Type":"ContainerStarted","Data":"9cb0e0f87351de4896381873d2889c3efc790869c70260c636a548e4544ac584"} Jul 13 20:14:52.594527 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:52.594503 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" event={"ID":"8396ce0a-7dc4-4e1f-aea0-32851b0c2402","Type":"ContainerStarted","Data":"1487d5c279c990a8b5cfab7d37f084b6a5a21e57edfc79cad8b00850d5268577"} Jul 13 20:14:53.397467 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:53.397434 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:53.397658 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:53.397434 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:53.397658 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:53.397563 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:53.397658 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:53.397440 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:53.397658 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:53.397638 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:53.397870 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:53.397691 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:54.603218 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:54.602973 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" event={"ID":"8396ce0a-7dc4-4e1f-aea0-32851b0c2402","Type":"ContainerStarted","Data":"9868cbd0c6a16c25d7bb682559f8dbb4076bf72e5e270a73c4df3139c6dbbca1"} Jul 13 20:14:54.609027 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:54.609007 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:14:54.609336 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:54.609317 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" event={"ID":"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae","Type":"ContainerStarted","Data":"a72cf50425719ece72ccdf20399af710b79f38e5b10b27625326430e84773c8e"} Jul 13 20:14:54.621611 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:54.621574 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-flnnl" podStartSLOduration=6.606310155 podStartE2EDuration="25.621561517s" totalImagesPullingTime="19.015251362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:52.609259922 +0000 UTC m=+23.755527093" watchObservedRunningTime="2026-07-13 20:14:54.621561517 +0000 UTC m=+25.767828701" Jul 13 20:14:54.621806 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:54.621789 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wzqv9" podStartSLOduration=3.743393121 podStartE2EDuration="25.621784322s" totalImagesPullingTime="21.878391201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:54.621405085 +0000 UTC m=+25.767672249" watchObservedRunningTime="2026-07-13 20:14:54.621784322 +0000 UTC m=+25.768051482" Jul 13 20:14:55.397139 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:55.397106 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:55.397303 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:55.397106 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:55.397303 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:55.397217 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:55.397303 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:55.397241 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:55.397410 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:55.397319 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:55.397410 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:55.397380 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:55.767443 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:55.767353 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:55.768101 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:55.767997 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:56.617116 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:56.616946 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:14:56.617473 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:56.617448 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" event={"ID":"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae","Type":"ContainerStarted","Data":"b04d15d4185ccec446a918889c2892d150b132547441ce63ee46c664fd7c8063"} Jul 13 20:14:56.617757 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:56.617739 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:56.617962 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:56.617945 2568 scope.go:117] "RemoveContainer" containerID="2c2613e316ec8f3060e76900913d4ed5b82e17a49ec001a4576e3b37c9fe85ef" Jul 13 20:14:56.618160 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:56.618145 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-plm24" Jul 13 20:14:57.397369 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:57.397334 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:57.397369 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:57.397362 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:57.398132 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:57.397333 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:57.398132 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:57.397492 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:57.398132 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:57.397547 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:57.399949 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:57.398933 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:57.624690 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:57.624661 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:14:57.625110 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:57.625078 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" event={"ID":"94fa8c97-cdf1-4b58-9fdb-5a4de75009ae","Type":"ContainerStarted","Data":"9881851ecbd11014f8ea79119d6e99103adf1afe85f827f6765606ecbdc51dfa"} Jul 13 20:14:57.625289 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:57.625262 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:57.625537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:57.625494 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:57.625647 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:57.625560 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:57.643313 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:57.643272 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:57.643650 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:57.643634 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:14:57.712286 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:57.712188 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" podStartSLOduration=9.612073033 podStartE2EDuration="28.712169375s" totalImagesPullingTime="19.100096342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:14:57.668156322 +0000 UTC m=+28.814423495" watchObservedRunningTime="2026-07-13 20:14:57.712169375 +0000 UTC m=+28.858436538" Jul 13 20:14:58.148155 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:58.148114 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bmv9n"] Jul 13 20:14:58.148310 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:58.148237 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:58.148360 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:58.148341 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:58.152331 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:58.152303 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jtdht"] Jul 13 20:14:58.152472 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:58.152408 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:14:58.152549 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:58.152478 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4hjb2"] Jul 13 20:14:58.152549 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:58.152523 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:14:58.152641 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:58.152549 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:58.152641 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:58.152622 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:59.014240 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:59.014208 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l8rnl_868f0604-458b-4ba7-a928-5cc701834300/dns-node-resolver/0.log" Jul 13 20:14:59.398481 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:59.398289 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:14:59.398721 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:59.398371 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:14:59.398721 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:59.398567 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:14:59.398721 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:14:59.398673 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:14:59.631469 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:59.631437 2568 generic.go:356] "Generic (PLEG): container finished" podID="d53e230c-cc46-44e1-a8f3-6cc9766fbf55" containerID="07343045ec91239317506741f3b4e81543bd4fa3e15e7ae2eb5080fc9596c5c8" exitCode=0 Jul 13 20:14:59.631613 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:59.631524 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z8ck" event={"ID":"d53e230c-cc46-44e1-a8f3-6cc9766fbf55","Type":"ContainerDied","Data":"07343045ec91239317506741f3b4e81543bd4fa3e15e7ae2eb5080fc9596c5c8"} Jul 13 20:14:59.980946 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:14:59.980876 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fdrjt_116a96e7-2d64-44a3-8e91-6681027a23ed/node-ca/0.log" Jul 13 20:15:00.397400 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:00.397380 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:00.397936 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:00.397494 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:00.635122 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:00.635084 2568 generic.go:356] "Generic (PLEG): container finished" podID="d53e230c-cc46-44e1-a8f3-6cc9766fbf55" containerID="878d0efce711d4ebbfd189578385a58cd2f392d32af621039c313d9f24513313" exitCode=0 Jul 13 20:15:00.635261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:00.635130 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z8ck" event={"ID":"d53e230c-cc46-44e1-a8f3-6cc9766fbf55","Type":"ContainerDied","Data":"878d0efce711d4ebbfd189578385a58cd2f392d32af621039c313d9f24513313"} Jul 13 20:15:01.397580 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:01.397554 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:01.398058 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:01.397565 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:01.398058 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:01.397673 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:15:01.398058 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:01.397745 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:15:02.397486 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:02.397451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:02.397633 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:02.397563 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:02.641490 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:02.641451 2568 generic.go:356] "Generic (PLEG): container finished" podID="d53e230c-cc46-44e1-a8f3-6cc9766fbf55" containerID="6f31235e63e610e9e0a38c7910587445484c8d8545b1b43d8ef1577a2234c20d" exitCode=0 Jul 13 20:15:02.641490 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:02.641493 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z8ck" event={"ID":"d53e230c-cc46-44e1-a8f3-6cc9766fbf55","Type":"ContainerDied","Data":"6f31235e63e610e9e0a38c7910587445484c8d8545b1b43d8ef1577a2234c20d"} Jul 13 20:15:03.016907 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:03.016819 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:03.017120 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:03.016935 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:15:03.017120 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:03.016987 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret podName:09d72f4e-4a0a-45c3-be16-f3c675544067 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:35.016973738 +0000 UTC m=+66.163240876 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret") pod "global-pull-secret-syncer-4hjb2" (UID: "09d72f4e-4a0a-45c3-be16-f3c675544067") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:15:03.117646 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:03.117614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:03.117816 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:03.117741 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:15:03.117816 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:03.117794 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs podName:eaff4f62-5b33-40cf-ab33-ea75668f2ebb nodeName:}" failed. No retries permitted until 2026-07-13 20:15:35.117782766 +0000 UTC m=+66.264049904 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs") pod "network-metrics-daemon-jtdht" (UID: "eaff4f62-5b33-40cf-ab33-ea75668f2ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:15:03.218488 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:03.218451 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngrct\" (UniqueName: \"kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct\") pod \"network-check-target-bmv9n\" (UID: \"94d176bf-61f1-47be-8342-638623aaffba\") " pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:03.218674 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:03.218633 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:15:03.218674 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:03.218665 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:15:03.218783 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:03.218679 2568 projected.go:196] Error preparing data for projected volume kube-api-access-ngrct for pod openshift-network-diagnostics/network-check-target-bmv9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:15:03.218783 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:03.218746 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct podName:94d176bf-61f1-47be-8342-638623aaffba nodeName:}" failed. No retries permitted until 2026-07-13 20:15:35.218725586 +0000 UTC m=+66.364992728 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ngrct" (UniqueName: "kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct") pod "network-check-target-bmv9n" (UID: "94d176bf-61f1-47be-8342-638623aaffba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:15:03.397722 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:03.397680 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:03.398197 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:03.397688 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:03.398197 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:03.397802 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:15:03.398197 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:03.397919 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:15:04.397433 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:04.397172 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:04.397619 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:04.397538 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:05.397746 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:05.397709 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:05.398171 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:05.397719 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:05.398171 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:05.397849 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:15:05.398171 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:05.397896 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:15:06.396779 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:06.396745 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:06.396963 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:06.396868 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:07.397202 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:07.397178 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:07.397590 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:07.397279 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:15:07.397590 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:07.397351 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:07.397590 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:07.397458 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:15:07.656880 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:07.656788 2568 generic.go:356] "Generic (PLEG): container finished" podID="d53e230c-cc46-44e1-a8f3-6cc9766fbf55" containerID="07770c2ec4750fed281bf3d9ee73b0eb6d3b1c4b230122cccb884bb999f36b1e" exitCode=0 Jul 13 20:15:07.657050 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:07.656866 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z8ck" event={"ID":"d53e230c-cc46-44e1-a8f3-6cc9766fbf55","Type":"ContainerDied","Data":"07770c2ec4750fed281bf3d9ee73b0eb6d3b1c4b230122cccb884bb999f36b1e"} Jul 13 20:15:08.397490 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:08.397454 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:08.397888 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:08.397578 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:08.661956 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:08.661874 2568 generic.go:356] "Generic (PLEG): container finished" podID="d53e230c-cc46-44e1-a8f3-6cc9766fbf55" containerID="946b4f9290a3e5c3c41f4874cff05e81102d5c2d4751b379c375f45250ca7153" exitCode=0 Jul 13 20:15:08.661956 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:08.661926 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z8ck" event={"ID":"d53e230c-cc46-44e1-a8f3-6cc9766fbf55","Type":"ContainerDied","Data":"946b4f9290a3e5c3c41f4874cff05e81102d5c2d4751b379c375f45250ca7153"} Jul 13 20:15:09.397910 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:09.397876 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:09.398269 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:09.397965 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:15:09.398269 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:09.398056 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:09.398269 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:09.398148 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:15:09.667349 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:09.667265 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z8ck" event={"ID":"d53e230c-cc46-44e1-a8f3-6cc9766fbf55","Type":"ContainerStarted","Data":"04f941c42b885af986e41ed2887d24a305a5cf94ac4931a71a56b0ce60f7e893"} Jul 13 20:15:09.692300 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:09.692233 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9z8ck" podStartSLOduration=8.14533209 podStartE2EDuration="40.692214775s" totalImagesPullingTime="32.546882685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:09.691799977 +0000 UTC m=+40.838067137" watchObservedRunningTime="2026-07-13 20:15:09.692214775 +0000 UTC m=+40.838481937" Jul 13 20:15:10.397432 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:10.397375 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:10.397634 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:10.397533 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:11.397058 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:11.397025 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:11.397471 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:11.397025 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:11.397471 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:11.397131 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:15:11.397471 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:11.397200 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:15:12.397256 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:12.397220 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:12.397656 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:12.397334 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:13.396734 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:13.396702 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:13.396734 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:13.396718 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:13.396985 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:13.396797 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:15:13.396985 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:13.396914 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:15:14.396731 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:14.396701 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:14.397128 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:14.396824 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:15.396943 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:15.396898 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:15.396943 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:15.396932 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:15.397348 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:15.397029 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:15:15.397348 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:15.397151 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:15:16.396943 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:16.396905 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:16.397113 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:16.397017 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:17.397552 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:17.397518 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:17.397928 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:17.397521 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:17.397928 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:17.397625 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:15:17.397928 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:17.397693 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:15:18.397289 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:18.397259 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:18.397486 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:18.397368 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:19.400697 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:19.398724 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:19.400697 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:19.398870 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:15:19.400697 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:19.399117 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:19.400697 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:19.399221 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:15:20.397067 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:20.397034 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:20.397272 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:20.397159 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:21.396744 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:21.396714 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:21.397251 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:21.396822 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:21.397251 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:21.396822 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4hjb2" podUID="09d72f4e-4a0a-45c3-be16-f3c675544067" Jul 13 20:15:21.397251 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:21.396910 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmv9n" podUID="94d176bf-61f1-47be-8342-638623aaffba" Jul 13 20:15:22.397013 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.396977 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:22.397483 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:22.397119 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jtdht" podUID="eaff4f62-5b33-40cf-ab33-ea75668f2ebb" Jul 13 20:15:22.717988 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.717913 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-149.ec2.internal" event="NodeReady" Jul 13 20:15:22.718133 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.718042 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 20:15:22.771301 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.771275 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-66tb9"] Jul 13 20:15:22.790083 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.790044 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8w4hv"] Jul 13 20:15:22.790255 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.790165 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.795946 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.793263 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jfzbg\"" Jul 13 20:15:22.795946 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.793690 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 20:15:22.795946 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.795623 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 20:15:22.805900 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.805880 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-66tb9"] Jul 13 20:15:22.806006 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.805918 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8w4hv"] Jul 13 20:15:22.806006 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.805949 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rmtq4"] Jul 13 20:15:22.806006 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.805998 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.809969 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.809946 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 20:15:22.810221 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.810207 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 20:15:22.810273 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.810207 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 20:15:22.810320 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.810286 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vgmz9\"" Jul 13 20:15:22.810900 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.810882 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 20:15:22.824062 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.824042 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rmtq4"] Jul 13 20:15:22.824166 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.824124 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rmtq4" Jul 13 20:15:22.830778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.830761 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 20:15:22.830897 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.830882 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2rtsn\"" Jul 13 20:15:22.831270 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.831245 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 20:15:22.833956 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.833943 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 20:15:22.863936 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.863910 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.863936 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.863937 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/958745c0-4511-4118-9b63-d62bf27a2317-config-volume\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.864118 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.863978 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkvh4\" (UniqueName: \"kubernetes.io/projected/fc7a2600-2ef7-4a68-94dd-d129973cf5e8-kube-api-access-tkvh4\") pod \"ingress-canary-rmtq4\" (UID: \"fc7a2600-2ef7-4a68-94dd-d129973cf5e8\") " pod="openshift-ingress-canary/ingress-canary-rmtq4" Jul 13 20:15:22.864118 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.864023 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-crio-socket\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.864118 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.864052 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-data-volume\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.864118 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.864093 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4np7n\" (UniqueName: \"kubernetes.io/projected/958745c0-4511-4118-9b63-d62bf27a2317-kube-api-access-4np7n\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.864317 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.864157 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/958745c0-4511-4118-9b63-d62bf27a2317-metrics-tls\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.864317 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.864194 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcssg\" (UniqueName: \"kubernetes.io/projected/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-kube-api-access-qcssg\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.864317 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.864226 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.864317 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.864265 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/958745c0-4511-4118-9b63-d62bf27a2317-tmp-dir\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.864498 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.864321 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc7a2600-2ef7-4a68-94dd-d129973cf5e8-cert\") pod \"ingress-canary-rmtq4\" (UID: \"fc7a2600-2ef7-4a68-94dd-d129973cf5e8\") " pod="openshift-ingress-canary/ingress-canary-rmtq4" Jul 13 20:15:22.965239 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965198 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.965239 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965236 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/958745c0-4511-4118-9b63-d62bf27a2317-config-volume\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.965549 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965268 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkvh4\" (UniqueName: \"kubernetes.io/projected/fc7a2600-2ef7-4a68-94dd-d129973cf5e8-kube-api-access-tkvh4\") pod \"ingress-canary-rmtq4\" (UID: \"fc7a2600-2ef7-4a68-94dd-d129973cf5e8\") " pod="openshift-ingress-canary/ingress-canary-rmtq4" Jul 13 20:15:22.965549 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965288 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-crio-socket\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.965549 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965347 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-crio-socket\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.965549 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965380 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-data-volume\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.965549 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965399 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4np7n\" (UniqueName: \"kubernetes.io/projected/958745c0-4511-4118-9b63-d62bf27a2317-kube-api-access-4np7n\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.965549 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/958745c0-4511-4118-9b63-d62bf27a2317-metrics-tls\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.965549 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965479 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcssg\" (UniqueName: \"kubernetes.io/projected/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-kube-api-access-qcssg\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.965888 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965694 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.965888 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965728 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/958745c0-4511-4118-9b63-d62bf27a2317-tmp-dir\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.965888 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965727 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-data-volume\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.965888 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965822 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc7a2600-2ef7-4a68-94dd-d129973cf5e8-cert\") pod \"ingress-canary-rmtq4\" (UID: \"fc7a2600-2ef7-4a68-94dd-d129973cf5e8\") " pod="openshift-ingress-canary/ingress-canary-rmtq4" Jul 13 20:15:22.966045 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965905 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.966045 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965933 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/958745c0-4511-4118-9b63-d62bf27a2317-config-volume\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.966045 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.965953 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/958745c0-4511-4118-9b63-d62bf27a2317-tmp-dir\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.969586 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.969532 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/958745c0-4511-4118-9b63-d62bf27a2317-metrics-tls\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.969586 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.969556 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:22.969712 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.969659 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc7a2600-2ef7-4a68-94dd-d129973cf5e8-cert\") pod \"ingress-canary-rmtq4\" (UID: \"fc7a2600-2ef7-4a68-94dd-d129973cf5e8\") " pod="openshift-ingress-canary/ingress-canary-rmtq4" Jul 13 20:15:22.990456 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.990410 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4np7n\" (UniqueName: \"kubernetes.io/projected/958745c0-4511-4118-9b63-d62bf27a2317-kube-api-access-4np7n\") pod \"dns-default-66tb9\" (UID: \"958745c0-4511-4118-9b63-d62bf27a2317\") " pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:22.990609 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.990585 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkvh4\" (UniqueName: \"kubernetes.io/projected/fc7a2600-2ef7-4a68-94dd-d129973cf5e8-kube-api-access-tkvh4\") pod \"ingress-canary-rmtq4\" (UID: \"fc7a2600-2ef7-4a68-94dd-d129973cf5e8\") " pod="openshift-ingress-canary/ingress-canary-rmtq4" Jul 13 20:15:22.991106 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:22.991086 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcssg\" (UniqueName: \"kubernetes.io/projected/cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50-kube-api-access-qcssg\") pod \"insights-runtime-extractor-8w4hv\" (UID: \"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50\") " pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:23.102402 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.102364 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:23.113485 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.113454 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8w4hv" Jul 13 20:15:23.131594 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.131562 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rmtq4" Jul 13 20:15:23.318275 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.318246 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-66tb9"] Jul 13 20:15:23.321519 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:23.321488 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958745c0_4511_4118_9b63_d62bf27a2317.slice/crio-95e31f13a047c7c94569a355e69a288271e80322c44baf002c064b1bc19ae402 WatchSource:0}: Error finding container 95e31f13a047c7c94569a355e69a288271e80322c44baf002c064b1bc19ae402: Status 404 returned error can't find the container with id 95e31f13a047c7c94569a355e69a288271e80322c44baf002c064b1bc19ae402 Jul 13 20:15:23.396897 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.396865 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:23.397064 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.396865 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:23.399225 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.399204 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 20:15:23.399542 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.399522 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 20:15:23.399542 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.399539 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 20:15:23.400044 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.400026 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fv6wc\"" Jul 13 20:15:23.538959 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.538882 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rmtq4"] Jul 13 20:15:23.542101 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.542076 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8w4hv"] Jul 13 20:15:23.542335 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:23.542223 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc7a2600_2ef7_4a68_94dd_d129973cf5e8.slice/crio-496db7da5982aa6bb797405846bca176c66fa246ed3024c6dba4e1c2b9fd2fb1 WatchSource:0}: Error finding container 496db7da5982aa6bb797405846bca176c66fa246ed3024c6dba4e1c2b9fd2fb1: Status 404 returned error can't find the container with id 496db7da5982aa6bb797405846bca176c66fa246ed3024c6dba4e1c2b9fd2fb1 Jul 13 20:15:23.544386 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:23.544367 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3ec07b_0adb_4ced_9fd5_619a3bf5ba50.slice/crio-1c7c4f02e898a5aab003d5500f9ddfea2c15ce257cce0c93c8ff25f1ee14cb87 WatchSource:0}: Error finding container 1c7c4f02e898a5aab003d5500f9ddfea2c15ce257cce0c93c8ff25f1ee14cb87: Status 404 returned error can't find the container with id 1c7c4f02e898a5aab003d5500f9ddfea2c15ce257cce0c93c8ff25f1ee14cb87 Jul 13 20:15:23.698578 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.698535 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rmtq4" event={"ID":"fc7a2600-2ef7-4a68-94dd-d129973cf5e8","Type":"ContainerStarted","Data":"496db7da5982aa6bb797405846bca176c66fa246ed3024c6dba4e1c2b9fd2fb1"} Jul 13 20:15:23.699623 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.699593 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8w4hv" event={"ID":"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50","Type":"ContainerStarted","Data":"1c7c4f02e898a5aab003d5500f9ddfea2c15ce257cce0c93c8ff25f1ee14cb87"} Jul 13 20:15:23.700545 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:23.700527 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-66tb9" event={"ID":"958745c0-4511-4118-9b63-d62bf27a2317","Type":"ContainerStarted","Data":"95e31f13a047c7c94569a355e69a288271e80322c44baf002c064b1bc19ae402"} Jul 13 20:15:24.397687 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:24.397650 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:24.400943 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:24.400912 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 20:15:24.401073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:24.400955 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qj7bp\"" Jul 13 20:15:27.395710 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.395678 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-q6lng"] Jul 13 20:15:27.415432 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.415381 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-q6lng"] Jul 13 20:15:27.415602 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.415495 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-q6lng" Jul 13 20:15:27.417974 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.417949 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-m2zpq\"" Jul 13 20:15:27.418364 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.418161 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 20:15:27.418364 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.418210 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 20:15:27.502789 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.502760 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v55g\" (UniqueName: \"kubernetes.io/projected/8a35a35a-0052-4fa5-9d36-bb8eb78b4c22-kube-api-access-2v55g\") pod \"downloads-68c9cbb497-q6lng\" (UID: \"8a35a35a-0052-4fa5-9d36-bb8eb78b4c22\") " pod="openshift-console/downloads-68c9cbb497-q6lng" Jul 13 20:15:27.510560 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.510528 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd"] Jul 13 20:15:27.513183 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.513160 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd" Jul 13 20:15:27.516871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.516854 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 13 20:15:27.518944 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.518923 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lz2c5\"" Jul 13 20:15:27.533611 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.533587 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd"] Jul 13 20:15:27.603366 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.603329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v55g\" (UniqueName: \"kubernetes.io/projected/8a35a35a-0052-4fa5-9d36-bb8eb78b4c22-kube-api-access-2v55g\") pod \"downloads-68c9cbb497-q6lng\" (UID: \"8a35a35a-0052-4fa5-9d36-bb8eb78b4c22\") " pod="openshift-console/downloads-68c9cbb497-q6lng" Jul 13 20:15:27.603566 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.603473 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/56f1bc0c-34ee-40c3-b9d7-e45ccf74413d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-68lbd\" (UID: \"56f1bc0c-34ee-40c3-b9d7-e45ccf74413d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd" Jul 13 20:15:27.612339 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.612313 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v55g\" (UniqueName: \"kubernetes.io/projected/8a35a35a-0052-4fa5-9d36-bb8eb78b4c22-kube-api-access-2v55g\") pod \"downloads-68c9cbb497-q6lng\" (UID: \"8a35a35a-0052-4fa5-9d36-bb8eb78b4c22\") " pod="openshift-console/downloads-68c9cbb497-q6lng" Jul 13 20:15:27.704513 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.704433 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/56f1bc0c-34ee-40c3-b9d7-e45ccf74413d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-68lbd\" (UID: \"56f1bc0c-34ee-40c3-b9d7-e45ccf74413d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd" Jul 13 20:15:27.704658 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:27.704630 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 13 20:15:27.704737 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:27.704723 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56f1bc0c-34ee-40c3-b9d7-e45ccf74413d-tls-certificates podName:56f1bc0c-34ee-40c3-b9d7-e45ccf74413d nodeName:}" failed. No retries permitted until 2026-07-13 20:15:28.204705383 +0000 UTC m=+59.350972526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/56f1bc0c-34ee-40c3-b9d7-e45ccf74413d-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-68lbd" (UID: "56f1bc0c-34ee-40c3-b9d7-e45ccf74413d") : secret "prometheus-operator-admission-webhook-tls" not found Jul 13 20:15:27.725821 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:27.725789 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-q6lng" Jul 13 20:15:28.209223 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:28.209195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/56f1bc0c-34ee-40c3-b9d7-e45ccf74413d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-68lbd\" (UID: \"56f1bc0c-34ee-40c3-b9d7-e45ccf74413d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd" Jul 13 20:15:28.211520 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:28.211498 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/56f1bc0c-34ee-40c3-b9d7-e45ccf74413d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-68lbd\" (UID: \"56f1bc0c-34ee-40c3-b9d7-e45ccf74413d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd" Jul 13 20:15:28.421873 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:28.421840 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd" Jul 13 20:15:29.254032 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.253938 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd"] Jul 13 20:15:29.272858 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:29.270817 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a35a35a_0052_4fa5_9d36_bb8eb78b4c22.slice/crio-207f6e2e403ac5e0b0970850e00c5d04ff2c1da430efa5a9607a5bb6cf98d3e5 WatchSource:0}: Error finding container 207f6e2e403ac5e0b0970850e00c5d04ff2c1da430efa5a9607a5bb6cf98d3e5: Status 404 returned error can't find the container with id 207f6e2e403ac5e0b0970850e00c5d04ff2c1da430efa5a9607a5bb6cf98d3e5 Jul 13 20:15:29.276731 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.276665 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-q6lng"] Jul 13 20:15:29.642225 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.642199 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nblwk" Jul 13 20:15:29.717007 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.716968 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rmtq4" event={"ID":"fc7a2600-2ef7-4a68-94dd-d129973cf5e8","Type":"ContainerStarted","Data":"45862b054ad12d0df773c844e5dc22e08ec31ce2b2bae11b72dfdb3b8c8ccbb4"} Jul 13 20:15:29.718511 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.718485 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8w4hv" event={"ID":"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50","Type":"ContainerStarted","Data":"caf6f5154b83c2cc47704249cacb63248aba748633de8501a4c5a089133529a4"} Jul 13 20:15:29.719486 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.719462 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd" event={"ID":"56f1bc0c-34ee-40c3-b9d7-e45ccf74413d","Type":"ContainerStarted","Data":"c1296b7306b10d5dfb04e45ceea8e54c98b6247842a9267460f594f2ff5f28d6"} Jul 13 20:15:29.720803 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.720775 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-q6lng" event={"ID":"8a35a35a-0052-4fa5-9d36-bb8eb78b4c22","Type":"ContainerStarted","Data":"207f6e2e403ac5e0b0970850e00c5d04ff2c1da430efa5a9607a5bb6cf98d3e5"} Jul 13 20:15:29.722453 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.722428 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-66tb9" event={"ID":"958745c0-4511-4118-9b63-d62bf27a2317","Type":"ContainerStarted","Data":"e1f1d6d43b6256663bce0805a7c43afcde78e074f12498021935ee160cf1fbf0"} Jul 13 20:15:29.722531 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.722464 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-66tb9" event={"ID":"958745c0-4511-4118-9b63-d62bf27a2317","Type":"ContainerStarted","Data":"b56856b40c59677e27501f1f4ed2704d737d9259191a80e3106a0055d42fcda0"} Jul 13 20:15:29.722573 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.722561 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:29.735769 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.735727 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rmtq4" podStartSLOduration=2.159108871 podStartE2EDuration="7.735714036s" totalImagesPullingTime="5.576605165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:29.734986958 +0000 UTC m=+60.881254133" watchObservedRunningTime="2026-07-13 20:15:29.735714036 +0000 UTC m=+60.881981188" Jul 13 20:15:29.755574 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:29.755532 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-66tb9" podStartSLOduration=1.986175794 podStartE2EDuration="7.755519075s" totalImagesPullingTime="5.769343281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:29.755308553 +0000 UTC m=+60.901575715" watchObservedRunningTime="2026-07-13 20:15:29.755519075 +0000 UTC m=+60.901786233" Jul 13 20:15:31.732432 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:31.732375 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8w4hv" event={"ID":"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50","Type":"ContainerStarted","Data":"19563da6dbd91fbaad6d7ce44e1f50c4d7b919767d6aaed827c88986bd9d1047"} Jul 13 20:15:32.737954 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:32.737915 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd" event={"ID":"56f1bc0c-34ee-40c3-b9d7-e45ccf74413d","Type":"ContainerStarted","Data":"0aea4772f499643c976963d9d9aa3cd4d67826e870b3ba5479478583f06d8da2"} Jul 13 20:15:32.738465 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:32.738125 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd" Jul 13 20:15:32.747970 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:32.747945 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd" Jul 13 20:15:32.754275 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:32.754221 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-68lbd" podStartSLOduration=3.038955442 podStartE2EDuration="5.754201854s" totalImagesPullingTime="2.715246412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:32.753455474 +0000 UTC m=+63.899722636" watchObservedRunningTime="2026-07-13 20:15:32.754201854 +0000 UTC m=+63.900469016" Jul 13 20:15:33.088260 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.088227 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59dd857f7c-ncjr8"] Jul 13 20:15:33.091938 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.091909 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.094628 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.094589 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 20:15:33.094758 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.094688 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 20:15:33.095043 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.094983 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 20:15:33.095622 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.095312 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 20:15:33.095622 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.095312 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 20:15:33.095622 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.095382 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jx2zn\"" Jul 13 20:15:33.101672 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.101653 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59dd857f7c-ncjr8"] Jul 13 20:15:33.146386 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.146354 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-serving-cert\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.146386 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.146397 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-service-ca\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.146650 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.146445 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-oauth-serving-cert\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.146650 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.146500 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-oauth-config\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.146650 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.146539 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-console-config\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.146650 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.146585 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvvpb\" (UniqueName: \"kubernetes.io/projected/bcb19aca-ffab-4246-ba24-09834c98f168-kube-api-access-bvvpb\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.247628 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.247586 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvvpb\" (UniqueName: \"kubernetes.io/projected/bcb19aca-ffab-4246-ba24-09834c98f168-kube-api-access-bvvpb\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.247828 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.247637 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-serving-cert\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.247828 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.247663 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-service-ca\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.247828 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.247689 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-oauth-serving-cert\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.247828 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.247742 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-oauth-config\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.247828 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.247782 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-console-config\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.248587 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.248531 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-console-config\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.248587 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.248554 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-service-ca\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.248780 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.248601 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-oauth-serving-cert\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.250521 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.250498 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-serving-cert\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.250783 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.250763 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-oauth-config\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.256860 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.256835 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvvpb\" (UniqueName: \"kubernetes.io/projected/bcb19aca-ffab-4246-ba24-09834c98f168-kube-api-access-bvvpb\") pod \"console-59dd857f7c-ncjr8\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.403043 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.402973 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:33.457020 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.456843 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-gj22q"] Jul 13 20:15:33.459260 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.459238 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:33.461882 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.461600 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 13 20:15:33.461882 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.461674 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-gjs8s\"" Jul 13 20:15:33.461882 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.461713 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 20:15:33.461882 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.461713 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 20:15:33.462176 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.461925 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 13 20:15:33.462176 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.461982 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 20:15:33.478122 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.478092 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-gj22q"] Jul 13 20:15:33.550865 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.550749 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:33.550865 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.550854 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvrpd\" (UniqueName: \"kubernetes.io/projected/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-kube-api-access-qvrpd\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:33.551125 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.550889 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-metrics-client-ca\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:33.551125 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.550925 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:33.652329 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.652284 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvrpd\" (UniqueName: \"kubernetes.io/projected/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-kube-api-access-qvrpd\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:33.652543 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.652334 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-metrics-client-ca\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:33.652543 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.652376 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:33.652543 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.652450 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:33.653271 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:33.652834 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 13 20:15:33.653271 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:33.652950 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-prometheus-operator-tls podName:8e2b8767-9ad0-42ec-bcb2-cfb2340fe190 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:34.152921664 +0000 UTC m=+65.299188815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-prometheus-operator-tls") pod "prometheus-operator-64785b6675-gj22q" (UID: "8e2b8767-9ad0-42ec-bcb2-cfb2340fe190") : secret "prometheus-operator-tls" not found Jul 13 20:15:33.653271 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.653235 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-metrics-client-ca\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:33.655458 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.655432 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:33.661668 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:33.661641 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvrpd\" (UniqueName: \"kubernetes.io/projected/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-kube-api-access-qvrpd\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:34.152912 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:34.152887 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59dd857f7c-ncjr8"] Jul 13 20:15:34.155715 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:34.155691 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:34.155815 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:34.155748 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb19aca_ffab_4246_ba24_09834c98f168.slice/crio-2ab307f729976ff18cda6ccb86e5836f56f465e91f2222369078064fdf07dc9b WatchSource:0}: Error finding container 2ab307f729976ff18cda6ccb86e5836f56f465e91f2222369078064fdf07dc9b: Status 404 returned error can't find the container with id 2ab307f729976ff18cda6ccb86e5836f56f465e91f2222369078064fdf07dc9b Jul 13 20:15:34.157962 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:34.157943 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e2b8767-9ad0-42ec-bcb2-cfb2340fe190-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-gj22q\" (UID: \"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:34.371201 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:34.371170 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" Jul 13 20:15:34.513393 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:34.513366 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-gj22q"] Jul 13 20:15:34.516118 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:34.516091 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2b8767_9ad0_42ec_bcb2_cfb2340fe190.slice/crio-080a58c9cc59bf10b2c7ee371f619781c32498ded3a32656640af7a8cb112763 WatchSource:0}: Error finding container 080a58c9cc59bf10b2c7ee371f619781c32498ded3a32656640af7a8cb112763: Status 404 returned error can't find the container with id 080a58c9cc59bf10b2c7ee371f619781c32498ded3a32656640af7a8cb112763 Jul 13 20:15:34.747617 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:34.747532 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8w4hv" event={"ID":"cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50","Type":"ContainerStarted","Data":"372bdd299039323d1d7f30575a9c8ed76fe2b1497d41752b026434e9ac2c472b"} Jul 13 20:15:34.748651 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:34.748628 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" event={"ID":"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190","Type":"ContainerStarted","Data":"080a58c9cc59bf10b2c7ee371f619781c32498ded3a32656640af7a8cb112763"} Jul 13 20:15:34.749689 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:34.749669 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dd857f7c-ncjr8" event={"ID":"bcb19aca-ffab-4246-ba24-09834c98f168","Type":"ContainerStarted","Data":"2ab307f729976ff18cda6ccb86e5836f56f465e91f2222369078064fdf07dc9b"} Jul 13 20:15:34.765208 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:34.765166 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8w4hv" podStartSLOduration=2.428229511 podStartE2EDuration="12.765153253s" totalImagesPullingTime="10.336923742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:15:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:34.764472409 +0000 UTC m=+65.910739569" watchObservedRunningTime="2026-07-13 20:15:34.765153253 +0000 UTC m=+65.911420392" Jul 13 20:15:35.063885 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.063839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:35.066367 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.066334 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 20:15:35.077030 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.077002 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/09d72f4e-4a0a-45c3-be16-f3c675544067-original-pull-secret\") pod \"global-pull-secret-syncer-4hjb2\" (UID: \"09d72f4e-4a0a-45c3-be16-f3c675544067\") " pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:35.112198 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.112165 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4hjb2" Jul 13 20:15:35.164837 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.164803 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:35.167749 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.167526 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 20:15:35.178111 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.178078 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaff4f62-5b33-40cf-ab33-ea75668f2ebb-metrics-certs\") pod \"network-metrics-daemon-jtdht\" (UID: \"eaff4f62-5b33-40cf-ab33-ea75668f2ebb\") " pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:35.210737 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.209192 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qj7bp\"" Jul 13 20:15:35.217302 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.217270 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jtdht" Jul 13 20:15:35.247119 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.246994 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4hjb2"] Jul 13 20:15:35.251582 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:35.251543 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09d72f4e_4a0a_45c3_be16_f3c675544067.slice/crio-19877e665c5cb4c93b16b937e1abceabdf88f9b2df686b616b6f9a34ded4f67c WatchSource:0}: Error finding container 19877e665c5cb4c93b16b937e1abceabdf88f9b2df686b616b6f9a34ded4f67c: Status 404 returned error can't find the container with id 19877e665c5cb4c93b16b937e1abceabdf88f9b2df686b616b6f9a34ded4f67c Jul 13 20:15:35.265722 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.265657 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngrct\" (UniqueName: \"kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct\") pod \"network-check-target-bmv9n\" (UID: \"94d176bf-61f1-47be-8342-638623aaffba\") " pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:35.269598 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.269350 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 20:15:35.278652 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.278610 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 20:15:35.290348 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.290288 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngrct\" (UniqueName: \"kubernetes.io/projected/94d176bf-61f1-47be-8342-638623aaffba-kube-api-access-ngrct\") pod \"network-check-target-bmv9n\" (UID: \"94d176bf-61f1-47be-8342-638623aaffba\") " pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:35.376189 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.376110 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jtdht"] Jul 13 20:15:35.379458 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:35.379406 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaff4f62_5b33_40cf_ab33_ea75668f2ebb.slice/crio-d14ae698a8a22fe5c8321115b9f27e03d61482b26d4c56c588bbd49e519648ee WatchSource:0}: Error finding container d14ae698a8a22fe5c8321115b9f27e03d61482b26d4c56c588bbd49e519648ee: Status 404 returned error can't find the container with id d14ae698a8a22fe5c8321115b9f27e03d61482b26d4c56c588bbd49e519648ee Jul 13 20:15:35.409571 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.409543 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fv6wc\"" Jul 13 20:15:35.416964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.416943 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:35.578105 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.578013 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bmv9n"] Jul 13 20:15:35.582707 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:35.582666 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d176bf_61f1_47be_8342_638623aaffba.slice/crio-a3d655346e9b4b814e5ba94ff94ba39bb5d9636d03db36f233fecb95dfc1a1a7 WatchSource:0}: Error finding container a3d655346e9b4b814e5ba94ff94ba39bb5d9636d03db36f233fecb95dfc1a1a7: Status 404 returned error can't find the container with id a3d655346e9b4b814e5ba94ff94ba39bb5d9636d03db36f233fecb95dfc1a1a7 Jul 13 20:15:35.755878 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.755833 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bmv9n" event={"ID":"94d176bf-61f1-47be-8342-638623aaffba","Type":"ContainerStarted","Data":"a3d655346e9b4b814e5ba94ff94ba39bb5d9636d03db36f233fecb95dfc1a1a7"} Jul 13 20:15:35.757213 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.757182 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4hjb2" event={"ID":"09d72f4e-4a0a-45c3-be16-f3c675544067","Type":"ContainerStarted","Data":"19877e665c5cb4c93b16b937e1abceabdf88f9b2df686b616b6f9a34ded4f67c"} Jul 13 20:15:35.759330 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:35.759308 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jtdht" event={"ID":"eaff4f62-5b33-40cf-ab33-ea75668f2ebb","Type":"ContainerStarted","Data":"d14ae698a8a22fe5c8321115b9f27e03d61482b26d4c56c588bbd49e519648ee"} Jul 13 20:15:39.728931 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:39.728730 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-66tb9" Jul 13 20:15:41.787205 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.787173 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jtdht" event={"ID":"eaff4f62-5b33-40cf-ab33-ea75668f2ebb","Type":"ContainerStarted","Data":"cf397d6de75d64c1fb41565c27169966f293884694b28ee6d90b2d0cabcf21cb"} Jul 13 20:15:41.787674 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.787228 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jtdht" event={"ID":"eaff4f62-5b33-40cf-ab33-ea75668f2ebb","Type":"ContainerStarted","Data":"1e48aae63e7f401a655f122a2068a0a937e9b1522442233b484667135991b593"} Jul 13 20:15:41.788953 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.788931 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" event={"ID":"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190","Type":"ContainerStarted","Data":"a3a1facae8f10c7bd2366aebd6f9055ae7453f454b1ceb4f8273ab86fe4e7d69"} Jul 13 20:15:41.789029 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.788972 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" event={"ID":"8e2b8767-9ad0-42ec-bcb2-cfb2340fe190","Type":"ContainerStarted","Data":"6c4d6d65fa023e6a92d5461736c62f016735eaa467baa8357cee33f18da0982d"} Jul 13 20:15:41.790459 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.790433 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dd857f7c-ncjr8" event={"ID":"bcb19aca-ffab-4246-ba24-09834c98f168","Type":"ContainerStarted","Data":"9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957"} Jul 13 20:15:41.792011 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.791992 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bmv9n" event={"ID":"94d176bf-61f1-47be-8342-638623aaffba","Type":"ContainerStarted","Data":"63f7f823c191319f4646cb67af7d34b2bd9d212314b35940f356438a69247ffd"} Jul 13 20:15:41.792107 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.792035 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:15:41.793318 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.793301 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4hjb2" event={"ID":"09d72f4e-4a0a-45c3-be16-f3c675544067","Type":"ContainerStarted","Data":"d0934a82245e2862d7cbcb9db739951d9613569c4426213b76e1a5e60b2c70c1"} Jul 13 20:15:41.804481 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.804442 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jtdht" podStartSLOduration=68.002717296 podStartE2EDuration="1m12.804431125s" totalImagesPullingTime="4.801713829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:41.804107636 +0000 UTC m=+72.950374871" watchObservedRunningTime="2026-07-13 20:15:41.804431125 +0000 UTC m=+72.950698305" Jul 13 20:15:41.818537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.818494 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4hjb2" podStartSLOduration=66.786247167 podStartE2EDuration="1m12.81848072s" totalImagesPullingTime="6.032233553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:41.817390673 +0000 UTC m=+72.963657833" watchObservedRunningTime="2026-07-13 20:15:41.81848072 +0000 UTC m=+72.964747881" Jul 13 20:15:41.833752 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.833688 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59dd857f7c-ncjr8" podStartSLOduration=1.726423242 podStartE2EDuration="8.833666971s" totalImagesPullingTime="7.107243729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:41.833226975 +0000 UTC m=+72.979494137" watchObservedRunningTime="2026-07-13 20:15:41.833666971 +0000 UTC m=+72.979934135" Jul 13 20:15:41.854169 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.854078 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-gj22q" podStartSLOduration=3.188231556 podStartE2EDuration="8.854055262s" totalImagesPullingTime="5.665823706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:41.85284343 +0000 UTC m=+72.999110628" watchObservedRunningTime="2026-07-13 20:15:41.854055262 +0000 UTC m=+73.000322409" Jul 13 20:15:41.870284 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:41.870220 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bmv9n" podStartSLOduration=67.169855882 podStartE2EDuration="1m12.870197102s" totalImagesPullingTime="5.70034122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:41.869273159 +0000 UTC m=+73.015540333" watchObservedRunningTime="2026-07-13 20:15:41.870197102 +0000 UTC m=+73.016464264" Jul 13 20:15:42.677560 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.677528 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64dfcb9fb7-mxst6"] Jul 13 20:15:42.680117 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.680098 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.692446 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.691966 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64dfcb9fb7-mxst6"] Jul 13 20:15:42.693339 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.693309 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 20:15:42.827244 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.827213 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-oauth-serving-cert\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.827639 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.827252 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-serving-cert\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.827639 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.827275 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-service-ca\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.827639 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.827295 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-console-config\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.827639 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.827351 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-trusted-ca-bundle\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.827639 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.827377 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-oauth-config\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.827639 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.827392 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l65j\" (UniqueName: \"kubernetes.io/projected/41c32431-d8cc-439f-8648-101f2735529e-kube-api-access-9l65j\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.928688 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.928603 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-service-ca\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.928688 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.928662 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-console-config\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.928957 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.928920 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-trusted-ca-bundle\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.929098 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.929042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-oauth-config\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.929098 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.929083 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l65j\" (UniqueName: \"kubernetes.io/projected/41c32431-d8cc-439f-8648-101f2735529e-kube-api-access-9l65j\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.929216 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.929133 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-oauth-serving-cert\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.929443 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.929273 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-serving-cert\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.929443 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.929397 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-service-ca\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.929577 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.929451 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-console-config\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.929825 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.929797 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-oauth-serving-cert\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.930606 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.930566 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-trusted-ca-bundle\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.931964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.931939 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-serving-cert\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.932175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.932155 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-oauth-config\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.937031 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.937014 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l65j\" (UniqueName: \"kubernetes.io/projected/41c32431-d8cc-439f-8648-101f2735529e-kube-api-access-9l65j\") pod \"console-64dfcb9fb7-mxst6\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:42.994942 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:42.994913 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:43.140108 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.140071 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64dfcb9fb7-mxst6"] Jul 13 20:15:43.145489 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:43.145462 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c32431_d8cc_439f_8648_101f2735529e.slice/crio-cf71a23d09d3f0c01e4e9b4c64521bb125786df3dfcf2e10d3852abd050d0df2 WatchSource:0}: Error finding container cf71a23d09d3f0c01e4e9b4c64521bb125786df3dfcf2e10d3852abd050d0df2: Status 404 returned error can't find the container with id cf71a23d09d3f0c01e4e9b4c64521bb125786df3dfcf2e10d3852abd050d0df2 Jul 13 20:15:43.403434 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.403384 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:43.403581 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.403506 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:43.408426 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.408387 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:43.807135 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.807104 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dfcb9fb7-mxst6" event={"ID":"41c32431-d8cc-439f-8648-101f2735529e","Type":"ContainerStarted","Data":"32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba"} Jul 13 20:15:43.807315 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.807156 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dfcb9fb7-mxst6" event={"ID":"41c32431-d8cc-439f-8648-101f2735529e","Type":"ContainerStarted","Data":"cf71a23d09d3f0c01e4e9b4c64521bb125786df3dfcf2e10d3852abd050d0df2"} Jul 13 20:15:43.811774 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.811726 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:15:43.813180 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.813156 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn"] Jul 13 20:15:43.815512 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.815493 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:43.817970 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.817952 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 13 20:15:43.818185 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.818159 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 13 20:15:43.818276 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.818258 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rzkdj\"" Jul 13 20:15:43.832327 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.832306 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn"] Jul 13 20:15:43.835091 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.835050 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64dfcb9fb7-mxst6" podStartSLOduration=1.8350358020000002 podStartE2EDuration="1.835035802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:43.834389516 +0000 UTC m=+74.980656678" watchObservedRunningTime="2026-07-13 20:15:43.835035802 +0000 UTC m=+74.981302963" Jul 13 20:15:43.852362 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.852340 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dqw5l"] Jul 13 20:15:43.854738 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.854721 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-txkzk"] Jul 13 20:15:43.854857 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.854838 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:43.857022 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.857000 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 20:15:43.857161 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.857141 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:43.857315 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.857266 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 20:15:43.857410 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.857320 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 20:15:43.857495 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.857431 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nkhzr\"" Jul 13 20:15:43.859654 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.859635 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 13 20:15:43.859753 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.859659 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bbsrh\"" Jul 13 20:15:43.859753 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.859677 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 13 20:15:43.859753 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.859706 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 13 20:15:43.866627 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.866608 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-txkzk"] Jul 13 20:15:43.936725 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.936691 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-textfile\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:43.936902 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.936790 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:43.936902 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.936839 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqzkz\" (UniqueName: \"kubernetes.io/projected/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-api-access-lqzkz\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:43.937708 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.937674 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02448df2-52e4-4456-8d7f-d5b22bbaaa90-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:43.937828 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.937739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-tls\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:43.937828 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.937769 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kndpq\" (UniqueName: \"kubernetes.io/projected/97e64137-f19c-4c25-9667-b62d069196aa-kube-api-access-kndpq\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938107 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97e64137-f19c-4c25-9667-b62d069196aa-sys\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938173 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938309 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97e64137-f19c-4c25-9667-b62d069196aa-root\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/785caab8-d70e-4b51-b02e-e3b54830e32d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938397 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/785caab8-d70e-4b51-b02e-e3b54830e32d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938450 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938485 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2pbh\" (UniqueName: \"kubernetes.io/projected/02448df2-52e4-4456-8d7f-d5b22bbaaa90-kube-api-access-c2pbh\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97e64137-f19c-4c25-9667-b62d069196aa-metrics-client-ca\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938617 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02448df2-52e4-4456-8d7f-d5b22bbaaa90-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938649 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/02448df2-52e4-4456-8d7f-d5b22bbaaa90-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:43.938778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:43.938723 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-wtmp\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.040402 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040366 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97e64137-f19c-4c25-9667-b62d069196aa-root\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.040605 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040410 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/785caab8-d70e-4b51-b02e-e3b54830e32d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.040605 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040460 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/785caab8-d70e-4b51-b02e-e3b54830e32d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.040605 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.040605 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040489 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97e64137-f19c-4c25-9667-b62d069196aa-root\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.040605 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040514 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2pbh\" (UniqueName: \"kubernetes.io/projected/02448df2-52e4-4456-8d7f-d5b22bbaaa90-kube-api-access-c2pbh\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:44.040872 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040612 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97e64137-f19c-4c25-9667-b62d069196aa-metrics-client-ca\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.040872 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040768 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02448df2-52e4-4456-8d7f-d5b22bbaaa90-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:44.040872 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040812 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.040872 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040849 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/02448df2-52e4-4456-8d7f-d5b22bbaaa90-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:44.041064 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040873 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/785caab8-d70e-4b51-b02e-e3b54830e32d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.041064 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040883 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-wtmp\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.041064 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040949 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-textfile\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.041064 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.040989 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.041064 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041023 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqzkz\" (UniqueName: \"kubernetes.io/projected/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-api-access-lqzkz\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.041064 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041056 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02448df2-52e4-4456-8d7f-d5b22bbaaa90-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:44.041408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041081 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-tls\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.041408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041105 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kndpq\" (UniqueName: \"kubernetes.io/projected/97e64137-f19c-4c25-9667-b62d069196aa-kube-api-access-kndpq\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.041408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041140 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97e64137-f19c-4c25-9667-b62d069196aa-sys\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.041408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041168 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.041408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041201 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.041408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041232 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97e64137-f19c-4c25-9667-b62d069196aa-metrics-client-ca\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.041408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041319 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/785caab8-d70e-4b51-b02e-e3b54830e32d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.041801 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041581 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-textfile\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.041801 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041709 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-wtmp\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.041801 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041713 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.042277 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.041965 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02448df2-52e4-4456-8d7f-d5b22bbaaa90-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:44.042277 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.042031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97e64137-f19c-4c25-9667-b62d069196aa-sys\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.042806 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.042757 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.043604 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.043561 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.043705 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.043658 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02448df2-52e4-4456-8d7f-d5b22bbaaa90-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:44.044487 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.044462 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/02448df2-52e4-4456-8d7f-d5b22bbaaa90-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:44.044605 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.044497 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.044671 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.044645 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97e64137-f19c-4c25-9667-b62d069196aa-node-exporter-tls\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.044724 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.044687 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.054560 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.054537 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kndpq\" (UniqueName: \"kubernetes.io/projected/97e64137-f19c-4c25-9667-b62d069196aa-kube-api-access-kndpq\") pod \"node-exporter-dqw5l\" (UID: \"97e64137-f19c-4c25-9667-b62d069196aa\") " pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.055172 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.055040 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqzkz\" (UniqueName: \"kubernetes.io/projected/785caab8-d70e-4b51-b02e-e3b54830e32d-kube-api-access-lqzkz\") pod \"kube-state-metrics-785778ff74-txkzk\" (UID: \"785caab8-d70e-4b51-b02e-e3b54830e32d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.055331 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.055191 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2pbh\" (UniqueName: \"kubernetes.io/projected/02448df2-52e4-4456-8d7f-d5b22bbaaa90-kube-api-access-c2pbh\") pod \"openshift-state-metrics-84bd68b896-p7kkn\" (UID: \"02448df2-52e4-4456-8d7f-d5b22bbaaa90\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:44.126962 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.126877 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" Jul 13 20:15:44.165008 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.164975 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dqw5l" Jul 13 20:15:44.171355 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.171334 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" Jul 13 20:15:44.866257 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.866225 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:15:44.871486 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.870222 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.872847 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.872826 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 20:15:44.872965 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.872896 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 20:15:44.873183 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.873150 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 20:15:44.873183 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.873176 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 20:15:44.873450 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.873254 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 20:15:44.873605 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.873590 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 20:15:44.873960 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.873945 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 20:15:44.874070 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.873959 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 20:15:44.874070 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.874008 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8gg6j\"" Jul 13 20:15:44.874253 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.874234 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 20:15:44.883954 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.883932 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:15:44.950011 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.949967 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-out\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950254 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950016 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950254 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950111 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950254 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950143 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-tls-assets\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950254 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950172 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8r7d\" (UniqueName: \"kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-kube-api-access-x8r7d\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950254 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950218 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950254 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950248 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950737 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950270 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950737 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950347 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950737 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950375 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950737 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950407 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950737 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950463 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-web-config\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:44.950737 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:44.950611 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-volume\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.051460 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051432 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-out\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.051632 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051476 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.051632 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051522 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.051632 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051547 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-tls-assets\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.051632 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051574 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8r7d\" (UniqueName: \"kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-kube-api-access-x8r7d\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.051632 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.051873 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051635 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.051873 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051658 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.051971 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051901 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.051971 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051937 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.051971 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051958 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.052119 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.051973 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.052119 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.052014 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-web-config\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.052119 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:45.052082 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-trusted-ca-bundle podName:95860f4c-3777-4ca1-9b33-4d6419c53b73 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:45.552061588 +0000 UTC m=+76.698328747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73") : configmap references non-existent config key: ca-bundle.crt Jul 13 20:15:45.052289 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.052136 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-volume\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.053199 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.053169 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.054798 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.054750 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-web-config\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.055232 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.055206 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.055609 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.055584 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-tls-assets\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.056001 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.055835 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-out\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.056001 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.055948 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.056243 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.056224 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.056343 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.056324 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.056673 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.056654 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.057804 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.057783 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-volume\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.060730 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.060709 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8r7d\" (UniqueName: \"kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-kube-api-access-x8r7d\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.556653 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.556600 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.557468 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.557438 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.782581 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.782545 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:15:45.890654 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.890569 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-867d97cf76-ljlbj"] Jul 13 20:15:45.896318 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.896290 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:45.899049 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.898980 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hcvcz\"" Jul 13 20:15:45.899049 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.898996 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 13 20:15:45.899049 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.899007 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 13 20:15:45.899049 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.899029 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 13 20:15:45.899329 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.898985 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1h1hmoisenrue\"" Jul 13 20:15:45.899663 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.899646 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 13 20:15:45.899770 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.899652 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 13 20:15:45.907114 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:45.907090 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-867d97cf76-ljlbj"] Jul 13 20:15:46.061583 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.061543 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35d75eb7-4b6f-4270-a7cc-d99f9291555e-metrics-client-ca\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.061761 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.061666 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.061761 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.061694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.061761 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.061716 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-tls\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.061896 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.061818 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.061896 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.061851 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.061896 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.061883 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-grpc-tls\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.062039 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.061942 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pdnq\" (UniqueName: \"kubernetes.io/projected/35d75eb7-4b6f-4270-a7cc-d99f9291555e-kube-api-access-9pdnq\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.163175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.163079 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.163175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.163127 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.163175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.163157 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-tls\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.163470 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.163210 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.163470 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.163240 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.163470 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.163262 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-grpc-tls\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.163470 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.163282 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pdnq\" (UniqueName: \"kubernetes.io/projected/35d75eb7-4b6f-4270-a7cc-d99f9291555e-kube-api-access-9pdnq\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.163470 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.163325 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35d75eb7-4b6f-4270-a7cc-d99f9291555e-metrics-client-ca\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.164122 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.164071 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35d75eb7-4b6f-4270-a7cc-d99f9291555e-metrics-client-ca\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.166370 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.166345 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.166495 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.166395 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.166543 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.166508 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.166715 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.166695 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-tls\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.166777 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.166743 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.167169 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.167146 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/35d75eb7-4b6f-4270-a7cc-d99f9291555e-secret-grpc-tls\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.171779 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.171754 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pdnq\" (UniqueName: \"kubernetes.io/projected/35d75eb7-4b6f-4270-a7cc-d99f9291555e-kube-api-access-9pdnq\") pod \"thanos-querier-867d97cf76-ljlbj\" (UID: \"35d75eb7-4b6f-4270-a7cc-d99f9291555e\") " pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:46.208536 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:46.208508 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:15:48.237090 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.237057 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs"] Jul 13 20:15:48.354363 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.354317 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs"] Jul 13 20:15:48.354555 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.354471 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.357461 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.357435 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 13 20:15:48.357612 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.357443 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vplrc\"" Jul 13 20:15:48.358435 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.358391 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 13 20:15:48.358564 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.358452 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 13 20:15:48.358564 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.358518 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-a6qgfskaqdods\"" Jul 13 20:15:48.358872 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.358853 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 13 20:15:48.485081 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.485045 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5pkf\" (UniqueName: \"kubernetes.io/projected/8f5e9872-59e3-4721-888c-999267360e49-kube-api-access-l5pkf\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.485277 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.485105 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8f5e9872-59e3-4721-888c-999267360e49-audit-log\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.485277 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.485170 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f5e9872-59e3-4721-888c-999267360e49-client-ca-bundle\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.485277 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.485211 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f5e9872-59e3-4721-888c-999267360e49-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.485277 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.485259 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8f5e9872-59e3-4721-888c-999267360e49-secret-metrics-server-client-certs\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.485488 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.485282 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8f5e9872-59e3-4721-888c-999267360e49-metrics-server-audit-profiles\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.485488 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.485309 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8f5e9872-59e3-4721-888c-999267360e49-secret-metrics-server-tls\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.585772 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.585732 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8f5e9872-59e3-4721-888c-999267360e49-secret-metrics-server-client-certs\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.585967 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.585785 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8f5e9872-59e3-4721-888c-999267360e49-metrics-server-audit-profiles\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.585967 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.585814 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8f5e9872-59e3-4721-888c-999267360e49-secret-metrics-server-tls\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.585967 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.585858 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5pkf\" (UniqueName: \"kubernetes.io/projected/8f5e9872-59e3-4721-888c-999267360e49-kube-api-access-l5pkf\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.585967 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.585914 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8f5e9872-59e3-4721-888c-999267360e49-audit-log\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.585967 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.585961 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f5e9872-59e3-4721-888c-999267360e49-client-ca-bundle\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.586294 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.586000 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f5e9872-59e3-4721-888c-999267360e49-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.586880 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.586858 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8f5e9872-59e3-4721-888c-999267360e49-metrics-server-audit-profiles\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.587109 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.587059 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8f5e9872-59e3-4721-888c-999267360e49-audit-log\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.588149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.588081 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f5e9872-59e3-4721-888c-999267360e49-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.589823 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.589781 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8f5e9872-59e3-4721-888c-999267360e49-secret-metrics-server-tls\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.590056 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.590031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f5e9872-59e3-4721-888c-999267360e49-client-ca-bundle\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.590170 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.590149 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8f5e9872-59e3-4721-888c-999267360e49-secret-metrics-server-client-certs\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.591086 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.591062 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz"] Jul 13 20:15:48.598639 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.598619 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5pkf\" (UniqueName: \"kubernetes.io/projected/8f5e9872-59e3-4721-888c-999267360e49-kube-api-access-l5pkf\") pod \"metrics-server-5c8d7d54c4-hxbvs\" (UID: \"8f5e9872-59e3-4721-888c-999267360e49\") " pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.607946 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.607896 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz"] Jul 13 20:15:48.608077 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.608006 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" Jul 13 20:15:48.610375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.610354 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 13 20:15:48.610507 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.610459 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-484d8\"" Jul 13 20:15:48.666387 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.666352 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:15:48.686986 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.686952 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb9n9\" (UniqueName: \"kubernetes.io/projected/852f60a6-e917-4e05-b249-b48a640f78e8-kube-api-access-mb9n9\") pod \"monitoring-plugin-5f68df7f58-qj4jz\" (UID: \"852f60a6-e917-4e05-b249-b48a640f78e8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" Jul 13 20:15:48.687115 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.687064 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/852f60a6-e917-4e05-b249-b48a640f78e8-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-qj4jz\" (UID: \"852f60a6-e917-4e05-b249-b48a640f78e8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" Jul 13 20:15:48.788253 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.788219 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb9n9\" (UniqueName: \"kubernetes.io/projected/852f60a6-e917-4e05-b249-b48a640f78e8-kube-api-access-mb9n9\") pod \"monitoring-plugin-5f68df7f58-qj4jz\" (UID: \"852f60a6-e917-4e05-b249-b48a640f78e8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" Jul 13 20:15:48.788473 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.788297 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/852f60a6-e917-4e05-b249-b48a640f78e8-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-qj4jz\" (UID: \"852f60a6-e917-4e05-b249-b48a640f78e8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" Jul 13 20:15:48.788473 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:48.788409 2568 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 13 20:15:48.788589 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:15:48.788486 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/852f60a6-e917-4e05-b249-b48a640f78e8-monitoring-plugin-cert podName:852f60a6-e917-4e05-b249-b48a640f78e8 nodeName:}" failed. No retries permitted until 2026-07-13 20:15:49.288465832 +0000 UTC m=+80.434732972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/852f60a6-e917-4e05-b249-b48a640f78e8-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-qj4jz" (UID: "852f60a6-e917-4e05-b249-b48a640f78e8") : secret "monitoring-plugin-cert" not found Jul 13 20:15:48.798931 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:48.798901 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb9n9\" (UniqueName: \"kubernetes.io/projected/852f60a6-e917-4e05-b249-b48a640f78e8-kube-api-access-mb9n9\") pod \"monitoring-plugin-5f68df7f58-qj4jz\" (UID: \"852f60a6-e917-4e05-b249-b48a640f78e8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" Jul 13 20:15:49.098962 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.098911 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64dfcb9fb7-mxst6"] Jul 13 20:15:49.129263 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.129234 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f7fc494c5-gck5b"] Jul 13 20:15:49.173989 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.173934 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f7fc494c5-gck5b"] Jul 13 20:15:49.174154 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.174055 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.292399 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.292343 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-oauth-config\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.292850 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.292409 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjqbj\" (UniqueName: \"kubernetes.io/projected/367c8878-77ed-4d3d-9105-23aef80fda79-kube-api-access-kjqbj\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.292850 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.292515 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-console-config\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.292850 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.292563 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-serving-cert\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.292850 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.292678 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-oauth-serving-cert\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.292850 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.292725 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-trusted-ca-bundle\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.292850 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.292756 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/852f60a6-e917-4e05-b249-b48a640f78e8-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-qj4jz\" (UID: \"852f60a6-e917-4e05-b249-b48a640f78e8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" Jul 13 20:15:49.292850 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.292843 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-service-ca\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.295370 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.295347 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/852f60a6-e917-4e05-b249-b48a640f78e8-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-qj4jz\" (UID: \"852f60a6-e917-4e05-b249-b48a640f78e8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" Jul 13 20:15:49.394292 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.394206 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-oauth-serving-cert\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.394292 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.394255 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-trusted-ca-bundle\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.394536 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.394319 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-service-ca\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.394536 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.394348 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-oauth-config\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.394536 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.394378 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjqbj\" (UniqueName: \"kubernetes.io/projected/367c8878-77ed-4d3d-9105-23aef80fda79-kube-api-access-kjqbj\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.394776 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.394749 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-console-config\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.394909 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.394800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-serving-cert\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.395122 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.395100 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-oauth-serving-cert\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.395222 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.395205 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-trusted-ca-bundle\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.395436 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.395384 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-service-ca\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.395537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.395469 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-console-config\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.397259 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.397236 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-oauth-config\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.397492 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.397474 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-serving-cert\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.402795 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.402772 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjqbj\" (UniqueName: \"kubernetes.io/projected/367c8878-77ed-4d3d-9105-23aef80fda79-kube-api-access-kjqbj\") pod \"console-6f7fc494c5-gck5b\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.485155 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.485127 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:49.519828 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:49.519803 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" Jul 13 20:15:50.085066 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.084927 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 20:15:50.105436 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.105324 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.108084 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.107907 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-x7qfj\"" Jul 13 20:15:50.109107 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.109078 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 13 20:15:50.109239 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.109149 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 13 20:15:50.109306 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.109084 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 13 20:15:50.110176 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.109522 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 13 20:15:50.110176 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.109794 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 13 20:15:50.110176 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.109929 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 13 20:15:50.110176 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.110062 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 13 20:15:50.110176 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.110130 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cs0bppg0flscg\"" Jul 13 20:15:50.110525 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.110321 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 13 20:15:50.110525 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.110369 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 13 20:15:50.110659 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.110597 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 13 20:15:50.110659 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.110610 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 13 20:15:50.110886 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.110865 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 13 20:15:50.111075 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.111056 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 13 20:15:50.111245 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.111226 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 20:15:50.113242 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.113202 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 13 20:15:50.201647 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201602 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config-out\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.201832 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201656 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.201832 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201697 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.201832 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201720 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.201832 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201743 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxrmg\" (UniqueName: \"kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-kube-api-access-sxrmg\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.201832 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201771 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.201832 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201792 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201838 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201859 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201883 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201910 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201939 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201967 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.201988 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-web-config\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.202023 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.202039 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.202062 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.202097 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.202118 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.202726 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.202150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.303483 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.303441 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.303950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.303496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.303950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.303704 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.303950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.303779 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config-out\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.303950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.303814 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.303950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.303852 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.303950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.303874 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.303950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.303898 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxrmg\" (UniqueName: \"kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-kube-api-access-sxrmg\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.303950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.303923 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.303950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.303951 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304009 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304033 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304059 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304095 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304116 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304145 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304181 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304207 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-web-config\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304269 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304296 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304321 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.304978 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304741 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.305271 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.305244 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.306980 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.306955 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.308432 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.307364 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.308432 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.307666 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.308432 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.308012 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.308432 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.308081 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.308432 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.308127 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.308432 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.308133 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.308432 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.304092 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.308832 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.308477 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.308832 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.308571 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.310879 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.310834 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.311085 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.311060 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-web-config\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.311156 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.311127 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config-out\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.313002 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.312976 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.313429 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.313397 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxrmg\" (UniqueName: \"kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-kube-api-access-sxrmg\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.314564 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.314542 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.315591 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.315570 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:50.423256 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:50.423162 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:15:51.666814 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:51.666782 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz"] Jul 13 20:15:51.693666 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:51.693551 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:15:51.764638 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:51.764584 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852f60a6_e917_4e05_b249_b48a640f78e8.slice/crio-3349bba7d47efb4b46890d83f45b8ebe68ec0483beaac0c9c059a20c21f33df6 WatchSource:0}: Error finding container 3349bba7d47efb4b46890d83f45b8ebe68ec0483beaac0c9c059a20c21f33df6: Status 404 returned error can't find the container with id 3349bba7d47efb4b46890d83f45b8ebe68ec0483beaac0c9c059a20c21f33df6 Jul 13 20:15:51.765357 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:51.765325 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95860f4c_3777_4ca1_9b33_4d6419c53b73.slice/crio-ab70640c4530c19123bc16158888f3dfdb62b3e460adecd907d81975f5c8f8e8 WatchSource:0}: Error finding container ab70640c4530c19123bc16158888f3dfdb62b3e460adecd907d81975f5c8f8e8: Status 404 returned error can't find the container with id ab70640c4530c19123bc16158888f3dfdb62b3e460adecd907d81975f5c8f8e8 Jul 13 20:15:51.849738 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:51.849707 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" event={"ID":"852f60a6-e917-4e05-b249-b48a640f78e8","Type":"ContainerStarted","Data":"3349bba7d47efb4b46890d83f45b8ebe68ec0483beaac0c9c059a20c21f33df6"} Jul 13 20:15:51.850843 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:51.850823 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dqw5l" event={"ID":"97e64137-f19c-4c25-9667-b62d069196aa","Type":"ContainerStarted","Data":"4e4fb2cf4814b7098d7dfbc12f24326189b43fc4143f79e3fd2c94d8ecab8fff"} Jul 13 20:15:51.851955 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:51.851935 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerStarted","Data":"ab70640c4530c19123bc16158888f3dfdb62b3e460adecd907d81975f5c8f8e8"} Jul 13 20:15:51.900457 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:51.900430 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn"] Jul 13 20:15:51.904256 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:51.904232 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-txkzk"] Jul 13 20:15:51.913403 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:51.913286 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02448df2_52e4_4456_8d7f_d5b22bbaaa90.slice/crio-33b73e8f587fd6547b7957af707995af3eabe22cede8c9de402a2822395eb453 WatchSource:0}: Error finding container 33b73e8f587fd6547b7957af707995af3eabe22cede8c9de402a2822395eb453: Status 404 returned error can't find the container with id 33b73e8f587fd6547b7957af707995af3eabe22cede8c9de402a2822395eb453 Jul 13 20:15:51.913707 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:51.913679 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785caab8_d70e_4b51_b02e_e3b54830e32d.slice/crio-dece385636e3f9b51e3890fd467ea4259630e0ead213cc893c9c320d848b8b30 WatchSource:0}: Error finding container dece385636e3f9b51e3890fd467ea4259630e0ead213cc893c9c320d848b8b30: Status 404 returned error can't find the container with id dece385636e3f9b51e3890fd467ea4259630e0ead213cc893c9c320d848b8b30 Jul 13 20:15:51.942667 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:51.942643 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f7fc494c5-gck5b"] Jul 13 20:15:51.946281 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:51.946227 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs"] Jul 13 20:15:51.948012 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:51.947982 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-867d97cf76-ljlbj"] Jul 13 20:15:51.948638 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:51.948614 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod367c8878_77ed_4d3d_9105_23aef80fda79.slice/crio-066935be03b340bc3aa94e55b0d844280ec3e18d496cb1e722d0bd9ee7d1a133 WatchSource:0}: Error finding container 066935be03b340bc3aa94e55b0d844280ec3e18d496cb1e722d0bd9ee7d1a133: Status 404 returned error can't find the container with id 066935be03b340bc3aa94e55b0d844280ec3e18d496cb1e722d0bd9ee7d1a133 Jul 13 20:15:51.950191 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:51.950016 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f5e9872_59e3_4721_888c_999267360e49.slice/crio-a4b30234bac7f5e50a7c8ec83d6436ba306e575eb3c40a79ee1c282c739dab22 WatchSource:0}: Error finding container a4b30234bac7f5e50a7c8ec83d6436ba306e575eb3c40a79ee1c282c739dab22: Status 404 returned error can't find the container with id a4b30234bac7f5e50a7c8ec83d6436ba306e575eb3c40a79ee1c282c739dab22 Jul 13 20:15:51.951468 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:51.951398 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35d75eb7_4b6f_4270_a7cc_d99f9291555e.slice/crio-eb17702dc13da246f03a34dcd37f04b687d177e20115acd55df2d891ca8596d6 WatchSource:0}: Error finding container eb17702dc13da246f03a34dcd37f04b687d177e20115acd55df2d891ca8596d6: Status 404 returned error can't find the container with id eb17702dc13da246f03a34dcd37f04b687d177e20115acd55df2d891ca8596d6 Jul 13 20:15:51.954258 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:51.954048 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 20:15:51.956382 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:15:51.956361 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c57bed8_a984_4b69_8f03_98f076b8bb4f.slice/crio-991144f10d24aab1b843bb3d68809200eea97f2969f12753921aaac3b8c73d1d WatchSource:0}: Error finding container 991144f10d24aab1b843bb3d68809200eea97f2969f12753921aaac3b8c73d1d: Status 404 returned error can't find the container with id 991144f10d24aab1b843bb3d68809200eea97f2969f12753921aaac3b8c73d1d Jul 13 20:15:52.863831 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.863783 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-q6lng" event={"ID":"8a35a35a-0052-4fa5-9d36-bb8eb78b4c22","Type":"ContainerStarted","Data":"23e58f03d1f4d2c38abcd00fb6a858bb93588a6b12656316b551de2b2ccaa03e"} Jul 13 20:15:52.864790 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.864770 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-q6lng" Jul 13 20:15:52.866836 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.866304 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerStarted","Data":"991144f10d24aab1b843bb3d68809200eea97f2969f12753921aaac3b8c73d1d"} Jul 13 20:15:52.868596 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.868561 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f7fc494c5-gck5b" event={"ID":"367c8878-77ed-4d3d-9105-23aef80fda79","Type":"ContainerStarted","Data":"f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5"} Jul 13 20:15:52.868737 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.868612 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f7fc494c5-gck5b" event={"ID":"367c8878-77ed-4d3d-9105-23aef80fda79","Type":"ContainerStarted","Data":"066935be03b340bc3aa94e55b0d844280ec3e18d496cb1e722d0bd9ee7d1a133"} Jul 13 20:15:52.870435 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.870393 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" event={"ID":"8f5e9872-59e3-4721-888c-999267360e49","Type":"ContainerStarted","Data":"a4b30234bac7f5e50a7c8ec83d6436ba306e575eb3c40a79ee1c282c739dab22"} Jul 13 20:15:52.871997 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.871975 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" event={"ID":"35d75eb7-4b6f-4270-a7cc-d99f9291555e","Type":"ContainerStarted","Data":"eb17702dc13da246f03a34dcd37f04b687d177e20115acd55df2d891ca8596d6"} Jul 13 20:15:52.875133 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.875109 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" event={"ID":"02448df2-52e4-4456-8d7f-d5b22bbaaa90","Type":"ContainerStarted","Data":"ad6a0c4d456f6dbefde5f5c619e5c26dd84d82966b9317768a5ea38393a28e9e"} Jul 13 20:15:52.875477 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.875457 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" event={"ID":"02448df2-52e4-4456-8d7f-d5b22bbaaa90","Type":"ContainerStarted","Data":"3c148fa52546a3d984f482635fc588f19b12444470dabbb82dc390e6825d803e"} Jul 13 20:15:52.875748 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.875728 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" event={"ID":"02448df2-52e4-4456-8d7f-d5b22bbaaa90","Type":"ContainerStarted","Data":"33b73e8f587fd6547b7957af707995af3eabe22cede8c9de402a2822395eb453"} Jul 13 20:15:52.877359 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.877316 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" event={"ID":"785caab8-d70e-4b51-b02e-e3b54830e32d","Type":"ContainerStarted","Data":"dece385636e3f9b51e3890fd467ea4259630e0ead213cc893c9c320d848b8b30"} Jul 13 20:15:52.878950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.878929 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-q6lng" Jul 13 20:15:52.881656 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.881611 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-q6lng" podStartSLOduration=3.324661354 podStartE2EDuration="25.881596468s" totalImagesPullingTime="22.556935114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:52.881308216 +0000 UTC m=+84.027575378" watchObservedRunningTime="2026-07-13 20:15:52.881596468 +0000 UTC m=+84.027863631" Jul 13 20:15:52.920173 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.918400 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f7fc494c5-gck5b" podStartSLOduration=3.9183805879999998 podStartE2EDuration="3.918380588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:52.918050138 +0000 UTC m=+84.064317312" watchObservedRunningTime="2026-07-13 20:15:52.918380588 +0000 UTC m=+84.064647752" Jul 13 20:15:52.995426 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:52.995379 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:15:53.887502 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:53.886944 2568 generic.go:356] "Generic (PLEG): container finished" podID="97e64137-f19c-4c25-9667-b62d069196aa" containerID="b2a2cb876fb70f7ecea6431cf58ffb8e44fb738892fe9312a084e62f885d74e2" exitCode=0 Jul 13 20:15:53.890012 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:53.888002 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dqw5l" event={"ID":"97e64137-f19c-4c25-9667-b62d069196aa","Type":"ContainerDied","Data":"b2a2cb876fb70f7ecea6431cf58ffb8e44fb738892fe9312a084e62f885d74e2"} Jul 13 20:15:57.909193 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:57.909109 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" event={"ID":"02448df2-52e4-4456-8d7f-d5b22bbaaa90","Type":"ContainerStarted","Data":"cca62e241e9c01b0c8516734d28ff46c8198081fdd4b08778c43265d5c1c6cee"} Jul 13 20:15:57.911758 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:57.911683 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerStarted","Data":"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f"} Jul 13 20:15:57.914953 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:57.914894 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dqw5l" event={"ID":"97e64137-f19c-4c25-9667-b62d069196aa","Type":"ContainerStarted","Data":"7ec7f62363a23fb790acc1fe5dff2319c6827ade2eee84f2c60522f890eb63ba"} Jul 13 20:15:57.930916 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:57.930856 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-p7kkn" podStartSLOduration=9.606738215 podStartE2EDuration="14.930837366s" totalImagesPullingTime="5.324099151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:57.930347349 +0000 UTC m=+89.076614510" watchObservedRunningTime="2026-07-13 20:15:57.930837366 +0000 UTC m=+89.077104528" Jul 13 20:15:58.923341 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.923249 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" event={"ID":"8f5e9872-59e3-4721-888c-999267360e49","Type":"ContainerStarted","Data":"f08619328d3e48e8f41b12047fe3b5d8043ea751d89b606c10488f94eebf761c"} Jul 13 20:15:58.928069 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.927900 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" event={"ID":"35d75eb7-4b6f-4270-a7cc-d99f9291555e","Type":"ContainerStarted","Data":"dd67fb810fba420f34e7d0f661bf5409affea755a7bc13149dd52fed94d8712f"} Jul 13 20:15:58.928069 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.927973 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" event={"ID":"35d75eb7-4b6f-4270-a7cc-d99f9291555e","Type":"ContainerStarted","Data":"dd22c7587720082a1c163929cfa0c1df84f6ff614e0d3b3e5e9adf30851799cd"} Jul 13 20:15:58.928069 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.928012 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" event={"ID":"35d75eb7-4b6f-4270-a7cc-d99f9291555e","Type":"ContainerStarted","Data":"1670378bf7da3a71fc02b58071c31d1046d815f3d6adf334008c71f7f613bce3"} Jul 13 20:15:58.930544 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.930515 2568 generic.go:356] "Generic (PLEG): container finished" podID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerID="2037e374e39c34282fc757729c70c19bd4a9ead99688cd629315a63fed5ae7df" exitCode=0 Jul 13 20:15:58.930683 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.930610 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerDied","Data":"2037e374e39c34282fc757729c70c19bd4a9ead99688cd629315a63fed5ae7df"} Jul 13 20:15:58.934480 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.934454 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" event={"ID":"785caab8-d70e-4b51-b02e-e3b54830e32d","Type":"ContainerStarted","Data":"ff6d8f6c62399ce2a9b050c31c6c499c1fec174d125190d1281fc4d89cefbeca"} Jul 13 20:15:58.934587 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.934526 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" event={"ID":"785caab8-d70e-4b51-b02e-e3b54830e32d","Type":"ContainerStarted","Data":"211bd7df1f961b236e1b0935ed9265c31d351a84826f4388cd80e36732b2d406"} Jul 13 20:15:58.934587 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.934557 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" event={"ID":"785caab8-d70e-4b51-b02e-e3b54830e32d","Type":"ContainerStarted","Data":"aa03258ce57d7f7c5ba3592bf757a97fc95932d90c67e06e21c265a8275ed6ca"} Jul 13 20:15:58.936929 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.936892 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" event={"ID":"852f60a6-e917-4e05-b249-b48a640f78e8","Type":"ContainerStarted","Data":"9a3693ea727ad044fb4dac1099373124f9f544618b1cec3369bd6c86a25a44fe"} Jul 13 20:15:58.937300 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.937279 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" Jul 13 20:15:58.940076 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.939195 2568 generic.go:356] "Generic (PLEG): container finished" podID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerID="45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f" exitCode=0 Jul 13 20:15:58.940076 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.939275 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerDied","Data":"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f"} Jul 13 20:15:58.944516 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.943779 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" podStartSLOduration=5.480352623 podStartE2EDuration="10.943752066s" totalImagesPullingTime="5.463399443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:58.941996663 +0000 UTC m=+90.088263825" watchObservedRunningTime="2026-07-13 20:15:58.943752066 +0000 UTC m=+90.090019230" Jul 13 20:15:58.946578 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.946556 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" Jul 13 20:15:58.953504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.953444 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dqw5l" event={"ID":"97e64137-f19c-4c25-9667-b62d069196aa","Type":"ContainerStarted","Data":"07f29ff60cf8eb4451bd6864f08509920eca3cdb5b4a3169cd9b0721d7ec2d98"} Jul 13 20:15:58.991656 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:58.991596 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-txkzk" podStartSLOduration=9.507980656 podStartE2EDuration="15.991573825s" totalImagesPullingTime="6.483593169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:58.961341272 +0000 UTC m=+90.107608432" watchObservedRunningTime="2026-07-13 20:15:58.991573825 +0000 UTC m=+90.137840989" Jul 13 20:15:59.013534 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:59.013472 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qj4jz" podStartSLOduration=5.376298138 podStartE2EDuration="11.013453514s" totalImagesPullingTime="5.637155376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:59.011623372 +0000 UTC m=+90.157890534" watchObservedRunningTime="2026-07-13 20:15:59.013453514 +0000 UTC m=+90.159720685" Jul 13 20:15:59.487711 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:59.487672 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:59.488737 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:59.488711 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:59.495851 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:59.495829 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:15:59.514669 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:59.514613 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dqw5l" podStartSLOduration=14.929194036 podStartE2EDuration="16.514594038s" totalImagesPullingTime="1.426542029s" totalInitContainerRuntime="158.857973ms" isStatefulPod=true podCreationTimestamp="2026-07-13 20:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:15:59.076329173 +0000 UTC m=+90.222596336" watchObservedRunningTime="2026-07-13 20:15:59.514594038 +0000 UTC m=+90.660861200" Jul 13 20:15:59.971975 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:15:59.971760 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:16:00.024911 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:00.024168 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59dd857f7c-ncjr8"] Jul 13 20:16:01.976560 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:01.976512 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" event={"ID":"35d75eb7-4b6f-4270-a7cc-d99f9291555e","Type":"ContainerStarted","Data":"463a2b1783d548b5104d132a08990a9bf851f474a504ecdad7a2a07acb6070b4"} Jul 13 20:16:01.977084 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:01.976627 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" event={"ID":"35d75eb7-4b6f-4270-a7cc-d99f9291555e","Type":"ContainerStarted","Data":"d927737f10b5eccd31678b050948c861f9b8b43cc66129fc4b3b88b101360495"} Jul 13 20:16:01.977084 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:01.976668 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" event={"ID":"35d75eb7-4b6f-4270-a7cc-d99f9291555e","Type":"ContainerStarted","Data":"82e52a944d40942481111a8c4db5910c47706cc3f7292204565e8c1d18219d21"} Jul 13 20:16:01.977084 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:01.976709 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:16:01.978716 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:01.978688 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerStarted","Data":"a4007b85b11a20d8ab2af51454fead9e8ef0e3c25cf8823075e86a6cd58c4561"} Jul 13 20:16:02.003825 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:02.003768 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" podStartSLOduration=8.18748494 podStartE2EDuration="17.003752535s" totalImagesPullingTime="8.816267595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:16:02.002727044 +0000 UTC m=+93.148994215" watchObservedRunningTime="2026-07-13 20:16:02.003752535 +0000 UTC m=+93.150019697" Jul 13 20:16:03.991238 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:03.991201 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerStarted","Data":"39a2544b60f5ecd81901bb4c405c7f1cc68ca601126fd69d1d2f24901c16623a"} Jul 13 20:16:03.991681 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:03.991287 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerStarted","Data":"c101393851f9914f45397a046803da50db57afac20742a4b80c47bc121d3e311"} Jul 13 20:16:03.991681 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:03.991357 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerStarted","Data":"0d819c6aa347c557ca45d0b792feee7852f348ba9fc7d76ed3f112cb446463e5"} Jul 13 20:16:03.991681 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:03.991397 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerStarted","Data":"2e26726bbcee9973dfc1682794a87e0679b90dd226b5c6c631aa8afed1d3919e"} Jul 13 20:16:03.991681 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:03.991456 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerStarted","Data":"0f18836b46c7f8075053bf086643b905d341e73523e073cd9057eb8cc48c4442"} Jul 13 20:16:03.994437 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:03.994399 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerStarted","Data":"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4"} Jul 13 20:16:03.994522 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:03.994467 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerStarted","Data":"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a"} Jul 13 20:16:03.994522 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:03.994494 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerStarted","Data":"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1"} Jul 13 20:16:03.994522 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:03.994517 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerStarted","Data":"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df"} Jul 13 20:16:03.994623 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:03.994540 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerStarted","Data":"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2"} Jul 13 20:16:04.017743 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:04.017688 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=11.590913793 podStartE2EDuration="20.017669874s" totalImagesPullingTime="8.391029949s" totalInitContainerRuntime="35.726132ms" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:16:04.016264749 +0000 UTC m=+95.162531923" watchObservedRunningTime="2026-07-13 20:16:04.017669874 +0000 UTC m=+95.163937037" Jul 13 20:16:05.003450 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:05.003389 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerStarted","Data":"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7"} Jul 13 20:16:05.034987 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:05.034921 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.130081982 podStartE2EDuration="15.034894325s" totalImagesPullingTime="9.883042055s" totalInitContainerRuntime="21.770288ms" isStatefulPod=false podCreationTimestamp="2026-07-13 20:15:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:16:05.0326979 +0000 UTC m=+96.178965062" watchObservedRunningTime="2026-07-13 20:16:05.034894325 +0000 UTC m=+96.181161488" Jul 13 20:16:05.423639 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:05.423602 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:16:07.989198 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:07.989124 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-867d97cf76-ljlbj" Jul 13 20:16:08.666968 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:08.666937 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:16:08.667153 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:08.666993 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:16:12.805067 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:12.805037 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bmv9n" Jul 13 20:16:14.121521 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.121459 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64dfcb9fb7-mxst6" podUID="41c32431-d8cc-439f-8648-101f2735529e" containerName="console" containerID="cri-o://32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba" gracePeriod=15 Jul 13 20:16:14.407202 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.407177 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:16:14.457271 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.457245 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-console-config\") pod \"41c32431-d8cc-439f-8648-101f2735529e\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " Jul 13 20:16:14.457446 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.457289 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l65j\" (UniqueName: \"kubernetes.io/projected/41c32431-d8cc-439f-8648-101f2735529e-kube-api-access-9l65j\") pod \"41c32431-d8cc-439f-8648-101f2735529e\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " Jul 13 20:16:14.457446 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.457331 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-oauth-serving-cert\") pod \"41c32431-d8cc-439f-8648-101f2735529e\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " Jul 13 20:16:14.457446 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.457369 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-oauth-config\") pod \"41c32431-d8cc-439f-8648-101f2735529e\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " Jul 13 20:16:14.457446 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.457395 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-service-ca\") pod \"41c32431-d8cc-439f-8648-101f2735529e\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " Jul 13 20:16:14.457642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.457485 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-trusted-ca-bundle\") pod \"41c32431-d8cc-439f-8648-101f2735529e\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " Jul 13 20:16:14.457642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.457512 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-serving-cert\") pod \"41c32431-d8cc-439f-8648-101f2735529e\" (UID: \"41c32431-d8cc-439f-8648-101f2735529e\") " Jul 13 20:16:14.457824 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.457770 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-console-config" (OuterVolumeSpecName: "console-config") pod "41c32431-d8cc-439f-8648-101f2735529e" (UID: "41c32431-d8cc-439f-8648-101f2735529e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:16:14.457824 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.457800 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "41c32431-d8cc-439f-8648-101f2735529e" (UID: "41c32431-d8cc-439f-8648-101f2735529e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:16:14.457972 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.457916 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-service-ca" (OuterVolumeSpecName: "service-ca") pod "41c32431-d8cc-439f-8648-101f2735529e" (UID: "41c32431-d8cc-439f-8648-101f2735529e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:16:14.457972 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.457927 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "41c32431-d8cc-439f-8648-101f2735529e" (UID: "41c32431-d8cc-439f-8648-101f2735529e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:16:14.459645 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.459622 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41c32431-d8cc-439f-8648-101f2735529e-kube-api-access-9l65j" (OuterVolumeSpecName: "kube-api-access-9l65j") pod "41c32431-d8cc-439f-8648-101f2735529e" (UID: "41c32431-d8cc-439f-8648-101f2735529e"). InnerVolumeSpecName "kube-api-access-9l65j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:16:14.459745 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.459677 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "41c32431-d8cc-439f-8648-101f2735529e" (UID: "41c32431-d8cc-439f-8648-101f2735529e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:16:14.459745 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.459708 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "41c32431-d8cc-439f-8648-101f2735529e" (UID: "41c32431-d8cc-439f-8648-101f2735529e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:16:14.558243 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.558209 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-oauth-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:14.558243 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.558234 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:14.558243 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.558245 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-trusted-ca-bundle\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:14.558243 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.558253 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41c32431-d8cc-439f-8648-101f2735529e-console-serving-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:14.558530 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.558262 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-console-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:14.558530 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.558270 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l65j\" (UniqueName: \"kubernetes.io/projected/41c32431-d8cc-439f-8648-101f2735529e-kube-api-access-9l65j\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:14.558530 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:14.558278 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41c32431-d8cc-439f-8648-101f2735529e-oauth-serving-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:15.049751 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:15.049714 2568 generic.go:356] "Generic (PLEG): container finished" podID="41c32431-d8cc-439f-8648-101f2735529e" containerID="32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba" exitCode=0 Jul 13 20:16:15.049939 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:15.049757 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dfcb9fb7-mxst6" Jul 13 20:16:15.049939 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:15.049781 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dfcb9fb7-mxst6" event={"ID":"41c32431-d8cc-439f-8648-101f2735529e","Type":"ContainerDied","Data":"32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba"} Jul 13 20:16:15.049939 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:15.049832 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dfcb9fb7-mxst6" event={"ID":"41c32431-d8cc-439f-8648-101f2735529e","Type":"ContainerDied","Data":"cf71a23d09d3f0c01e4e9b4c64521bb125786df3dfcf2e10d3852abd050d0df2"} Jul 13 20:16:15.049939 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:15.049858 2568 scope.go:117] "RemoveContainer" containerID="32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba" Jul 13 20:16:15.062171 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:15.062151 2568 scope.go:117] "RemoveContainer" containerID="32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba" Jul 13 20:16:15.062457 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:16:15.062434 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba\": container with ID starting with 32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba not found: ID does not exist" containerID="32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba" Jul 13 20:16:15.062512 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:15.062467 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba"} err="failed to get container status \"32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba\": rpc error: code = NotFound desc = could not find container \"32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba\": container with ID starting with 32dca94d77d8085afae30d3d2b1fd3b19c5c20b5b6746fb63effe7bcf6a40fba not found: ID does not exist" Jul 13 20:16:15.072574 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:15.072550 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64dfcb9fb7-mxst6"] Jul 13 20:16:15.078172 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:15.078149 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64dfcb9fb7-mxst6"] Jul 13 20:16:15.402906 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:15.402826 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41c32431-d8cc-439f-8648-101f2735529e" path="/var/lib/kubelet/pods/41c32431-d8cc-439f-8648-101f2735529e/volumes" Jul 13 20:16:25.059246 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.059204 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59dd857f7c-ncjr8" podUID="bcb19aca-ffab-4246-ba24-09834c98f168" containerName="console" containerID="cri-o://9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957" gracePeriod=15 Jul 13 20:16:25.363551 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.363523 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:16:25.451439 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.449687 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-oauth-config\") pod \"bcb19aca-ffab-4246-ba24-09834c98f168\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " Jul 13 20:16:25.451439 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.449764 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvvpb\" (UniqueName: \"kubernetes.io/projected/bcb19aca-ffab-4246-ba24-09834c98f168-kube-api-access-bvvpb\") pod \"bcb19aca-ffab-4246-ba24-09834c98f168\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " Jul 13 20:16:25.451439 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.449829 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-service-ca\") pod \"bcb19aca-ffab-4246-ba24-09834c98f168\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " Jul 13 20:16:25.451439 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.449870 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-oauth-serving-cert\") pod \"bcb19aca-ffab-4246-ba24-09834c98f168\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " Jul 13 20:16:25.451439 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.449911 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-console-config\") pod \"bcb19aca-ffab-4246-ba24-09834c98f168\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " Jul 13 20:16:25.451439 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.449977 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-serving-cert\") pod \"bcb19aca-ffab-4246-ba24-09834c98f168\" (UID: \"bcb19aca-ffab-4246-ba24-09834c98f168\") " Jul 13 20:16:25.455435 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.451956 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-service-ca" (OuterVolumeSpecName: "service-ca") pod "bcb19aca-ffab-4246-ba24-09834c98f168" (UID: "bcb19aca-ffab-4246-ba24-09834c98f168"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:16:25.455435 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.452192 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bcb19aca-ffab-4246-ba24-09834c98f168" (UID: "bcb19aca-ffab-4246-ba24-09834c98f168"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:16:25.455435 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.452465 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-console-config" (OuterVolumeSpecName: "console-config") pod "bcb19aca-ffab-4246-ba24-09834c98f168" (UID: "bcb19aca-ffab-4246-ba24-09834c98f168"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:16:25.458439 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.455852 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bcb19aca-ffab-4246-ba24-09834c98f168" (UID: "bcb19aca-ffab-4246-ba24-09834c98f168"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:16:25.464340 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.464311 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb19aca-ffab-4246-ba24-09834c98f168-kube-api-access-bvvpb" (OuterVolumeSpecName: "kube-api-access-bvvpb") pod "bcb19aca-ffab-4246-ba24-09834c98f168" (UID: "bcb19aca-ffab-4246-ba24-09834c98f168"). InnerVolumeSpecName "kube-api-access-bvvpb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:16:25.464586 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.464566 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bcb19aca-ffab-4246-ba24-09834c98f168" (UID: "bcb19aca-ffab-4246-ba24-09834c98f168"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:16:25.551111 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.551076 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvvpb\" (UniqueName: \"kubernetes.io/projected/bcb19aca-ffab-4246-ba24-09834c98f168-kube-api-access-bvvpb\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:25.551111 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.551114 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:25.551290 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.551130 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-oauth-serving-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:25.551290 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.551144 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bcb19aca-ffab-4246-ba24-09834c98f168-console-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:25.551290 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.551157 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-serving-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:25.551290 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:25.551170 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bcb19aca-ffab-4246-ba24-09834c98f168-console-oauth-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:16:26.105043 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:26.105007 2568 generic.go:356] "Generic (PLEG): container finished" podID="bcb19aca-ffab-4246-ba24-09834c98f168" containerID="9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957" exitCode=0 Jul 13 20:16:26.105538 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:26.105068 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dd857f7c-ncjr8" Jul 13 20:16:26.105538 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:26.105101 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dd857f7c-ncjr8" event={"ID":"bcb19aca-ffab-4246-ba24-09834c98f168","Type":"ContainerDied","Data":"9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957"} Jul 13 20:16:26.105538 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:26.105181 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dd857f7c-ncjr8" event={"ID":"bcb19aca-ffab-4246-ba24-09834c98f168","Type":"ContainerDied","Data":"2ab307f729976ff18cda6ccb86e5836f56f465e91f2222369078064fdf07dc9b"} Jul 13 20:16:26.105538 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:26.105201 2568 scope.go:117] "RemoveContainer" containerID="9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957" Jul 13 20:16:26.118575 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:26.118547 2568 scope.go:117] "RemoveContainer" containerID="9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957" Jul 13 20:16:26.119983 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:16:26.119957 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957\": container with ID starting with 9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957 not found: ID does not exist" containerID="9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957" Jul 13 20:16:26.120139 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:26.120113 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957"} err="failed to get container status \"9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957\": rpc error: code = NotFound desc = could not find container \"9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957\": container with ID starting with 9e4be8417305214f34e6fa9be76c276754139a22754cc1cbabc321bb7e301957 not found: ID does not exist" Jul 13 20:16:26.129828 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:26.129800 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59dd857f7c-ncjr8"] Jul 13 20:16:26.133106 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:26.133064 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59dd857f7c-ncjr8"] Jul 13 20:16:27.402661 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:27.402627 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcb19aca-ffab-4246-ba24-09834c98f168" path="/var/lib/kubelet/pods/bcb19aca-ffab-4246-ba24-09834c98f168/volumes" Jul 13 20:16:28.671790 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:28.671762 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:16:28.675651 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:28.675635 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c8d7d54c4-hxbvs" Jul 13 20:16:50.423438 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:50.423367 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:16:50.443550 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:50.443523 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:16:51.223591 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:16:51.223568 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:04.129877 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.129842 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:17:04.130442 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.130210 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy" containerID="cri-o://0d819c6aa347c557ca45d0b792feee7852f348ba9fc7d76ed3f112cb446463e5" gracePeriod=120 Jul 13 20:17:04.130442 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.130281 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="prom-label-proxy" containerID="cri-o://39a2544b60f5ecd81901bb4c405c7f1cc68ca601126fd69d1d2f24901c16623a" gracePeriod=120 Jul 13 20:17:04.130442 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.130368 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy-metric" containerID="cri-o://c101393851f9914f45397a046803da50db57afac20742a4b80c47bc121d3e311" gracePeriod=120 Jul 13 20:17:04.130442 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.130280 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy-web" containerID="cri-o://2e26726bbcee9973dfc1682794a87e0679b90dd226b5c6c631aa8afed1d3919e" gracePeriod=120 Jul 13 20:17:04.130646 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.130340 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="config-reloader" containerID="cri-o://0f18836b46c7f8075053bf086643b905d341e73523e073cd9057eb8cc48c4442" gracePeriod=120 Jul 13 20:17:04.130646 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.130197 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="alertmanager" containerID="cri-o://a4007b85b11a20d8ab2af51454fead9e8ef0e3c25cf8823075e86a6cd58c4561" gracePeriod=120 Jul 13 20:17:04.263607 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.263582 2568 generic.go:356] "Generic (PLEG): container finished" podID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerID="39a2544b60f5ecd81901bb4c405c7f1cc68ca601126fd69d1d2f24901c16623a" exitCode=0 Jul 13 20:17:04.263607 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.263605 2568 generic.go:356] "Generic (PLEG): container finished" podID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerID="c101393851f9914f45397a046803da50db57afac20742a4b80c47bc121d3e311" exitCode=0 Jul 13 20:17:04.263743 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.263613 2568 generic.go:356] "Generic (PLEG): container finished" podID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerID="0d819c6aa347c557ca45d0b792feee7852f348ba9fc7d76ed3f112cb446463e5" exitCode=0 Jul 13 20:17:04.263743 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.263620 2568 generic.go:356] "Generic (PLEG): container finished" podID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerID="0f18836b46c7f8075053bf086643b905d341e73523e073cd9057eb8cc48c4442" exitCode=0 Jul 13 20:17:04.263743 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.263628 2568 generic.go:356] "Generic (PLEG): container finished" podID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerID="a4007b85b11a20d8ab2af51454fead9e8ef0e3c25cf8823075e86a6cd58c4561" exitCode=0 Jul 13 20:17:04.263743 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.263651 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerDied","Data":"39a2544b60f5ecd81901bb4c405c7f1cc68ca601126fd69d1d2f24901c16623a"} Jul 13 20:17:04.263743 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.263714 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerDied","Data":"c101393851f9914f45397a046803da50db57afac20742a4b80c47bc121d3e311"} Jul 13 20:17:04.263743 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.263741 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerDied","Data":"0d819c6aa347c557ca45d0b792feee7852f348ba9fc7d76ed3f112cb446463e5"} Jul 13 20:17:04.263922 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.263766 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerDied","Data":"0f18836b46c7f8075053bf086643b905d341e73523e073cd9057eb8cc48c4442"} Jul 13 20:17:04.263922 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:04.263792 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerDied","Data":"a4007b85b11a20d8ab2af51454fead9e8ef0e3c25cf8823075e86a6cd58c4561"} Jul 13 20:17:05.273045 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.273010 2568 generic.go:356] "Generic (PLEG): container finished" podID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerID="2e26726bbcee9973dfc1682794a87e0679b90dd226b5c6c631aa8afed1d3919e" exitCode=0 Jul 13 20:17:05.273386 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.273078 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerDied","Data":"2e26726bbcee9973dfc1682794a87e0679b90dd226b5c6c631aa8afed1d3919e"} Jul 13 20:17:05.377196 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.377172 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:05.510239 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510150 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-metric\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510239 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510214 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-main-db\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510239 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510236 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-volume\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510560 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510337 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-tls-assets\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510560 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510383 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-web-config\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510560 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510438 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-cluster-tls-config\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510728 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510582 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:17:05.510728 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510585 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8r7d\" (UniqueName: \"kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-kube-api-access-x8r7d\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510728 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510688 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-main-tls\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510728 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510724 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510924 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510770 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-metrics-client-ca\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510924 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510807 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-trusted-ca-bundle\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510924 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510844 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-web\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.510924 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.510882 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-out\") pod \"95860f4c-3777-4ca1-9b33-4d6419c53b73\" (UID: \"95860f4c-3777-4ca1-9b33-4d6419c53b73\") " Jul 13 20:17:05.511178 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.511158 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-main-db\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.511826 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.511793 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:05.513031 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.512942 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-volume" (OuterVolumeSpecName: "config-volume") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:05.513591 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.513560 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:17:05.513688 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.513626 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-out" (OuterVolumeSpecName: "config-out") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:17:05.513688 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.513662 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:05.513688 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.513684 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-kube-api-access-x8r7d" (OuterVolumeSpecName: "kube-api-access-x8r7d") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "kube-api-access-x8r7d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:17:05.514117 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.514070 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:05.514226 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.514162 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:05.515590 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.515563 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:05.515703 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.515598 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:05.518773 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.518403 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:05.525554 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.525521 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-web-config" (OuterVolumeSpecName: "web-config") pod "95860f4c-3777-4ca1-9b33-4d6419c53b73" (UID: "95860f4c-3777-4ca1-9b33-4d6419c53b73"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:05.612149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612114 2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-volume\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.612149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612143 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-tls-assets\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.612149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612151 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-web-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.612149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612159 2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-cluster-tls-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.612428 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612169 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8r7d\" (UniqueName: \"kubernetes.io/projected/95860f4c-3777-4ca1-9b33-4d6419c53b73-kube-api-access-x8r7d\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.612428 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612179 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-main-tls\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.612428 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612189 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.612428 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612198 2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-metrics-client-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.612428 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612206 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95860f4c-3777-4ca1-9b33-4d6419c53b73-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.612428 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612215 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.612428 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612223 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/95860f4c-3777-4ca1-9b33-4d6419c53b73-config-out\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:05.612428 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:05.612234 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/95860f4c-3777-4ca1-9b33-4d6419c53b73-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:06.280283 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.280245 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95860f4c-3777-4ca1-9b33-4d6419c53b73","Type":"ContainerDied","Data":"ab70640c4530c19123bc16158888f3dfdb62b3e460adecd907d81975f5c8f8e8"} Jul 13 20:17:06.280283 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.280275 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.280716 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.280343 2568 scope.go:117] "RemoveContainer" containerID="39a2544b60f5ecd81901bb4c405c7f1cc68ca601126fd69d1d2f24901c16623a" Jul 13 20:17:06.289160 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.289141 2568 scope.go:117] "RemoveContainer" containerID="c101393851f9914f45397a046803da50db57afac20742a4b80c47bc121d3e311" Jul 13 20:17:06.296057 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.296041 2568 scope.go:117] "RemoveContainer" containerID="0d819c6aa347c557ca45d0b792feee7852f348ba9fc7d76ed3f112cb446463e5" Jul 13 20:17:06.303030 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.303009 2568 scope.go:117] "RemoveContainer" containerID="2e26726bbcee9973dfc1682794a87e0679b90dd226b5c6c631aa8afed1d3919e" Jul 13 20:17:06.304200 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.304180 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:17:06.307320 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.307302 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:17:06.313089 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.313061 2568 scope.go:117] "RemoveContainer" containerID="0f18836b46c7f8075053bf086643b905d341e73523e073cd9057eb8cc48c4442" Jul 13 20:17:06.320006 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.319985 2568 scope.go:117] "RemoveContainer" containerID="a4007b85b11a20d8ab2af51454fead9e8ef0e3c25cf8823075e86a6cd58c4561" Jul 13 20:17:06.327223 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.327206 2568 scope.go:117] "RemoveContainer" containerID="2037e374e39c34282fc757729c70c19bd4a9ead99688cd629315a63fed5ae7df" Jul 13 20:17:06.332689 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.332670 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:17:06.332961 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.332950 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy-web" Jul 13 20:17:06.333001 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.332963 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy-web" Jul 13 20:17:06.333001 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.332976 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41c32431-d8cc-439f-8648-101f2735529e" containerName="console" Jul 13 20:17:06.333001 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.332982 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41c32431-d8cc-439f-8648-101f2735529e" containerName="console" Jul 13 20:17:06.333001 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.332988 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="init-config-reloader" Jul 13 20:17:06.333001 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.332993 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="init-config-reloader" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333003 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="alertmanager" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333009 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="alertmanager" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333021 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy-metric" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333027 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy-metric" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333034 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb19aca-ffab-4246-ba24-09834c98f168" containerName="console" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333039 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb19aca-ffab-4246-ba24-09834c98f168" containerName="console" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333047 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="config-reloader" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333052 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="config-reloader" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333059 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333065 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333072 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="prom-label-proxy" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333078 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="prom-label-proxy" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333129 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb19aca-ffab-4246-ba24-09834c98f168" containerName="console" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333141 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="alertmanager" Jul 13 20:17:06.333149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333149 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="prom-label-proxy" Jul 13 20:17:06.333700 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333159 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="41c32431-d8cc-439f-8648-101f2735529e" containerName="console" Jul 13 20:17:06.333700 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333168 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy-metric" Jul 13 20:17:06.333700 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333179 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="config-reloader" Jul 13 20:17:06.333700 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333187 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy-web" Jul 13 20:17:06.333700 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.333197 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" containerName="kube-rbac-proxy" Jul 13 20:17:06.337706 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.337689 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.340225 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.340203 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 20:17:06.340315 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.340228 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 20:17:06.340360 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.340307 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 20:17:06.340514 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.340502 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 20:17:06.340689 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.340673 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 20:17:06.341063 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.341046 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 20:17:06.341154 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.341141 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8gg6j\"" Jul 13 20:17:06.341386 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.341370 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 20:17:06.341479 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.341443 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 20:17:06.346005 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.345986 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 20:17:06.353534 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.353511 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:17:06.419488 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419448 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blzmf\" (UniqueName: \"kubernetes.io/projected/81974efb-ad6d-4280-a040-5aa6e357721d-kube-api-access-blzmf\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419657 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419509 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419657 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419657 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419573 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419657 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-web-config\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419657 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419617 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419657 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419644 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81974efb-ad6d-4280-a040-5aa6e357721d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419934 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419701 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/81974efb-ad6d-4280-a040-5aa6e357721d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419934 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419731 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81974efb-ad6d-4280-a040-5aa6e357721d-config-out\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419934 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419814 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81974efb-ad6d-4280-a040-5aa6e357721d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419934 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419847 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419934 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81974efb-ad6d-4280-a040-5aa6e357721d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.419934 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.419897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-config-volume\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521168 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521203 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521230 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521247 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-web-config\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521595 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521272 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521595 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521292 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81974efb-ad6d-4280-a040-5aa6e357721d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521595 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521313 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/81974efb-ad6d-4280-a040-5aa6e357721d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521595 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521340 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81974efb-ad6d-4280-a040-5aa6e357721d-config-out\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521595 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521379 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81974efb-ad6d-4280-a040-5aa6e357721d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521595 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521404 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521595 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521467 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81974efb-ad6d-4280-a040-5aa6e357721d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521595 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-config-volume\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.521595 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.521552 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blzmf\" (UniqueName: \"kubernetes.io/projected/81974efb-ad6d-4280-a040-5aa6e357721d-kube-api-access-blzmf\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.522544 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.522435 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81974efb-ad6d-4280-a040-5aa6e357721d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.523010 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.522724 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/81974efb-ad6d-4280-a040-5aa6e357721d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.524621 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.524286 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.524621 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.524286 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-web-config\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.524621 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.524444 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.524621 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.524575 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.524621 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.524605 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.524900 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.524625 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.524994 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.524969 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81974efb-ad6d-4280-a040-5aa6e357721d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.525166 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.525147 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81974efb-ad6d-4280-a040-5aa6e357721d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.525231 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.525175 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81974efb-ad6d-4280-a040-5aa6e357721d-config-out\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.526205 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.526189 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/81974efb-ad6d-4280-a040-5aa6e357721d-config-volume\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.530138 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.530120 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blzmf\" (UniqueName: \"kubernetes.io/projected/81974efb-ad6d-4280-a040-5aa6e357721d-kube-api-access-blzmf\") pod \"alertmanager-main-0\" (UID: \"81974efb-ad6d-4280-a040-5aa6e357721d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.647388 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.647339 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:17:06.784644 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:06.784618 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:17:06.786426 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:17:06.786385 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81974efb_ad6d_4280_a040_5aa6e357721d.slice/crio-6e03085d1c9dd2bdf318f56a6d62dd121f331727303f5cb565bc35b4c6890e0c WatchSource:0}: Error finding container 6e03085d1c9dd2bdf318f56a6d62dd121f331727303f5cb565bc35b4c6890e0c: Status 404 returned error can't find the container with id 6e03085d1c9dd2bdf318f56a6d62dd121f331727303f5cb565bc35b4c6890e0c Jul 13 20:17:07.287377 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:07.287342 2568 generic.go:356] "Generic (PLEG): container finished" podID="81974efb-ad6d-4280-a040-5aa6e357721d" containerID="ff6ed4b5d5a1ed2fb340df27d6b41616c56da035ad89cdb7903d7ba1f80f6208" exitCode=0 Jul 13 20:17:07.287852 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:07.287436 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81974efb-ad6d-4280-a040-5aa6e357721d","Type":"ContainerDied","Data":"ff6ed4b5d5a1ed2fb340df27d6b41616c56da035ad89cdb7903d7ba1f80f6208"} Jul 13 20:17:07.287852 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:07.287526 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81974efb-ad6d-4280-a040-5aa6e357721d","Type":"ContainerStarted","Data":"6e03085d1c9dd2bdf318f56a6d62dd121f331727303f5cb565bc35b4c6890e0c"} Jul 13 20:17:07.402852 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:07.402828 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95860f4c-3777-4ca1-9b33-4d6419c53b73" path="/var/lib/kubelet/pods/95860f4c-3777-4ca1-9b33-4d6419c53b73/volumes" Jul 13 20:17:08.182504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.182470 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b547fd5f-mmcvv"] Jul 13 20:17:08.185183 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.185167 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.187946 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.187698 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 20:17:08.187946 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.187917 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-7kq2q\"" Jul 13 20:17:08.188145 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.188106 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 20:17:08.188249 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.188225 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 20:17:08.188312 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.188253 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 20:17:08.188312 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.188302 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 20:17:08.193397 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.193364 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 20:17:08.206440 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.204862 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b547fd5f-mmcvv"] Jul 13 20:17:08.238601 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.238568 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-federate-client-tls\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.238784 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.238630 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-serving-certs-ca-bundle\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.238784 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.238663 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-secret-telemeter-client\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.238784 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.238694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-metrics-client-ca\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.238784 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.238774 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.238964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.238804 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.238964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.238892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnshj\" (UniqueName: \"kubernetes.io/projected/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-kube-api-access-qnshj\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.238964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.238932 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-telemeter-client-tls\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.295646 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.295609 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81974efb-ad6d-4280-a040-5aa6e357721d","Type":"ContainerStarted","Data":"6425d701fad9fff778715e528e75e66f8122fd0b5601d06ea5893f0ec37abc59"} Jul 13 20:17:08.296048 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.295674 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81974efb-ad6d-4280-a040-5aa6e357721d","Type":"ContainerStarted","Data":"50debffbade1d897d183db40bdd5648eb6e6beb744ff1251b9b5cfd4c974e579"} Jul 13 20:17:08.296048 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.295700 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81974efb-ad6d-4280-a040-5aa6e357721d","Type":"ContainerStarted","Data":"175173512b9716001c8803786465af036d454a58ff1b58eb6283b1463743a03a"} Jul 13 20:17:08.296048 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.295726 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81974efb-ad6d-4280-a040-5aa6e357721d","Type":"ContainerStarted","Data":"0742d46f8092d4a5d932c166d3986552d2e80d69b7c19f2648fcf1c111ec5c84"} Jul 13 20:17:08.296048 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.295749 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81974efb-ad6d-4280-a040-5aa6e357721d","Type":"ContainerStarted","Data":"045294ecea5feb8eda3db5f29fbfd4f35cf9a18b234a5631d598170a794c5e57"} Jul 13 20:17:08.296048 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.295777 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81974efb-ad6d-4280-a040-5aa6e357721d","Type":"ContainerStarted","Data":"dde92eb2cddff9ad600e9fe63558c7b1eff7fb3a56ec6e87275ef87cae7f5444"} Jul 13 20:17:08.339160 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.339094 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.32611727 podStartE2EDuration="2.339078041s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.960771ms" isStatefulPod=false podCreationTimestamp="2026-07-13 20:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:17:08.337126584 +0000 UTC m=+159.483393748" watchObservedRunningTime="2026-07-13 20:17:08.339078041 +0000 UTC m=+159.485345202" Jul 13 20:17:08.339795 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.339766 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.339935 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.339802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.339935 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.339826 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnshj\" (UniqueName: \"kubernetes.io/projected/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-kube-api-access-qnshj\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.339935 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.339844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-telemeter-client-tls\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.339935 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.339887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-federate-client-tls\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.339935 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.339924 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-serving-certs-ca-bundle\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.340178 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.339949 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-secret-telemeter-client\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.340178 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.339978 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-metrics-client-ca\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.340852 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.340828 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-metrics-client-ca\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.341001 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.340973 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.341277 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.341244 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-serving-certs-ca-bundle\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.342630 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.342608 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-secret-telemeter-client\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.342742 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.342720 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-telemeter-client-tls\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.343075 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.343054 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.343134 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.343083 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-federate-client-tls\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.350514 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.350487 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnshj\" (UniqueName: \"kubernetes.io/projected/0c2f3ae3-90de-403b-b7fd-050f4ec77f47-kube-api-access-qnshj\") pod \"telemeter-client-7b547fd5f-mmcvv\" (UID: \"0c2f3ae3-90de-403b-b7fd-050f4ec77f47\") " pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.497650 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.497565 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" Jul 13 20:17:08.530108 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.530053 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 20:17:08.530674 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.530462 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="prometheus" containerID="cri-o://b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2" gracePeriod=600 Jul 13 20:17:08.530674 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.530525 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="config-reloader" containerID="cri-o://0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df" gracePeriod=600 Jul 13 20:17:08.530674 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.530553 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy-web" containerID="cri-o://2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a" gracePeriod=600 Jul 13 20:17:08.530674 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.530472 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7" gracePeriod=600 Jul 13 20:17:08.530674 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.530553 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="thanos-sidecar" containerID="cri-o://3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1" gracePeriod=600 Jul 13 20:17:08.530674 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.530555 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy" containerID="cri-o://c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4" gracePeriod=600 Jul 13 20:17:08.672253 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.672052 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b547fd5f-mmcvv"] Jul 13 20:17:08.675907 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:17:08.675879 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c2f3ae3_90de_403b_b7fd_050f4ec77f47.slice/crio-44c70c36e976d9f87ed42b80c4dcba1c012a6f61df4ea6d9751792d4d71ac4cc WatchSource:0}: Error finding container 44c70c36e976d9f87ed42b80c4dcba1c012a6f61df4ea6d9751792d4d71ac4cc: Status 404 returned error can't find the container with id 44c70c36e976d9f87ed42b80c4dcba1c012a6f61df4ea6d9751792d4d71ac4cc Jul 13 20:17:08.794978 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.794953 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:08.946297 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946258 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-kube-rbac-proxy\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.946297 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946298 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config-out\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.946642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946327 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-kubelet-serving-ca-bundle\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.946642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946363 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-serving-certs-ca-bundle\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.946642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946382 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.946642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946401 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.946642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946457 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxrmg\" (UniqueName: \"kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-kube-api-access-sxrmg\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.946642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946494 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-metrics-client-ca\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.946642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946518 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-2\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.946642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946544 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-1\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.946642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946587 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-web-config\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.946642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946625 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946658 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-db\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946686 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-trusted-ca-bundle\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946721 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-tls-assets\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946754 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-thanos-prometheus-http-client-file\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946750 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946786 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-0\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946783 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946838 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-tls\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946863 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-grpc-tls\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.946889 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-metrics-client-certs\") pod \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\" (UID: \"4c57bed8-a984-4b69-8f03-98f076b8bb4f\") " Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.947040 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:08.947123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.947064 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:08.947717 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.947210 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:08.947717 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.947229 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:08.947717 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.947245 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-configmap-metrics-client-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:08.947717 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.947259 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:08.947717 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.947410 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:08.947717 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.947663 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:08.948610 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.948177 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:17:08.949207 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.948921 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:08.949207 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.949113 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:08.949544 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.949500 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config-out" (OuterVolumeSpecName: "config-out") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:17:08.950076 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.950050 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:17:08.950620 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.950586 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-kube-api-access-sxrmg" (OuterVolumeSpecName: "kube-api-access-sxrmg") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "kube-api-access-sxrmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:17:08.950750 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.950619 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:08.951054 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.951032 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:08.951124 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.951051 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:08.951584 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.951560 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:08.951767 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.951747 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:08.951899 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.951880 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:08.952394 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.952381 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config" (OuterVolumeSpecName: "config") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:08.961865 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:08.961842 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-web-config" (OuterVolumeSpecName: "web-config") pod "4c57bed8-a984-4b69-8f03-98f076b8bb4f" (UID: "4c57bed8-a984-4b69-8f03-98f076b8bb4f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:09.048116 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048087 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048116 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048115 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-db\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048124 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-tls-assets\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048133 2568 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048144 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048154 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048163 2568 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-grpc-tls\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048172 2568 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-metrics-client-certs\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048181 2568 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-kube-rbac-proxy\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048188 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config-out\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048196 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048206 2568 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048214 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxrmg\" (UniqueName: \"kubernetes.io/projected/4c57bed8-a984-4b69-8f03-98f076b8bb4f-kube-api-access-sxrmg\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048222 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048230 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4c57bed8-a984-4b69-8f03-98f076b8bb4f-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.048299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.048238 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c57bed8-a984-4b69-8f03-98f076b8bb4f-web-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:09.303626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303532 2568 generic.go:356] "Generic (PLEG): container finished" podID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerID="83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7" exitCode=0 Jul 13 20:17:09.303626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303554 2568 generic.go:356] "Generic (PLEG): container finished" podID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerID="c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4" exitCode=0 Jul 13 20:17:09.303626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303561 2568 generic.go:356] "Generic (PLEG): container finished" podID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerID="2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a" exitCode=0 Jul 13 20:17:09.303626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303567 2568 generic.go:356] "Generic (PLEG): container finished" podID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerID="3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1" exitCode=0 Jul 13 20:17:09.303626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303572 2568 generic.go:356] "Generic (PLEG): container finished" podID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerID="0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df" exitCode=0 Jul 13 20:17:09.303626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303576 2568 generic.go:356] "Generic (PLEG): container finished" podID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerID="b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2" exitCode=0 Jul 13 20:17:09.303626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303616 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerDied","Data":"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7"} Jul 13 20:17:09.304468 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303637 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.304468 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303691 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerDied","Data":"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4"} Jul 13 20:17:09.304468 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303741 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerDied","Data":"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a"} Jul 13 20:17:09.304468 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303764 2568 scope.go:117] "RemoveContainer" containerID="83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7" Jul 13 20:17:09.304468 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303786 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerDied","Data":"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1"} Jul 13 20:17:09.304468 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303828 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerDied","Data":"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df"} Jul 13 20:17:09.304468 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303866 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerDied","Data":"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2"} Jul 13 20:17:09.304468 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.303897 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c57bed8-a984-4b69-8f03-98f076b8bb4f","Type":"ContainerDied","Data":"991144f10d24aab1b843bb3d68809200eea97f2969f12753921aaac3b8c73d1d"} Jul 13 20:17:09.305051 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.305026 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" event={"ID":"0c2f3ae3-90de-403b-b7fd-050f4ec77f47","Type":"ContainerStarted","Data":"44c70c36e976d9f87ed42b80c4dcba1c012a6f61df4ea6d9751792d4d71ac4cc"} Jul 13 20:17:09.312973 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.312955 2568 scope.go:117] "RemoveContainer" containerID="c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4" Jul 13 20:17:09.322731 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.322164 2568 scope.go:117] "RemoveContainer" containerID="2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a" Jul 13 20:17:09.330206 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.330172 2568 scope.go:117] "RemoveContainer" containerID="3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1" Jul 13 20:17:09.332481 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.332454 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 20:17:09.337157 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.337137 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 20:17:09.339574 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.339550 2568 scope.go:117] "RemoveContainer" containerID="0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df" Jul 13 20:17:09.349621 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.349581 2568 scope.go:117] "RemoveContainer" containerID="b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2" Jul 13 20:17:09.359112 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.359090 2568 scope.go:117] "RemoveContainer" containerID="45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f" Jul 13 20:17:09.364550 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.364516 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 20:17:09.365046 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365022 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy-thanos" Jul 13 20:17:09.365046 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365049 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy-thanos" Jul 13 20:17:09.365175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365070 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="prometheus" Jul 13 20:17:09.365175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365080 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="prometheus" Jul 13 20:17:09.365175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365093 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy" Jul 13 20:17:09.365175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365104 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy" Jul 13 20:17:09.365175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365118 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="thanos-sidecar" Jul 13 20:17:09.365175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365127 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="thanos-sidecar" Jul 13 20:17:09.365175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365141 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="config-reloader" Jul 13 20:17:09.365175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365149 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="config-reloader" Jul 13 20:17:09.365175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365168 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="init-config-reloader" Jul 13 20:17:09.365175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365176 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="init-config-reloader" Jul 13 20:17:09.365518 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365185 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy-web" Jul 13 20:17:09.365518 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365193 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy-web" Jul 13 20:17:09.365518 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365273 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy-thanos" Jul 13 20:17:09.365518 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365286 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="thanos-sidecar" Jul 13 20:17:09.365518 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365296 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="prometheus" Jul 13 20:17:09.365518 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365305 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="config-reloader" Jul 13 20:17:09.365518 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365316 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy" Jul 13 20:17:09.365518 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.365328 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" containerName="kube-rbac-proxy-web" Jul 13 20:17:09.368587 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.368570 2568 scope.go:117] "RemoveContainer" containerID="83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7" Jul 13 20:17:09.368909 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:17:09.368873 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": container with ID starting with 83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7 not found: ID does not exist" containerID="83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7" Jul 13 20:17:09.368991 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.368911 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7"} err="failed to get container status \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": rpc error: code = NotFound desc = could not find container \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": container with ID starting with 83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7 not found: ID does not exist" Jul 13 20:17:09.368991 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.368937 2568 scope.go:117] "RemoveContainer" containerID="c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4" Jul 13 20:17:09.369237 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:17:09.369213 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": container with ID starting with c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4 not found: ID does not exist" containerID="c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4" Jul 13 20:17:09.369303 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.369246 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4"} err="failed to get container status \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": rpc error: code = NotFound desc = could not find container \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": container with ID starting with c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4 not found: ID does not exist" Jul 13 20:17:09.369303 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.369267 2568 scope.go:117] "RemoveContainer" containerID="2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a" Jul 13 20:17:09.369595 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:17:09.369570 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": container with ID starting with 2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a not found: ID does not exist" containerID="2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a" Jul 13 20:17:09.369678 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.369604 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a"} err="failed to get container status \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": rpc error: code = NotFound desc = could not find container \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": container with ID starting with 2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a not found: ID does not exist" Jul 13 20:17:09.369678 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.369625 2568 scope.go:117] "RemoveContainer" containerID="3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1" Jul 13 20:17:09.369832 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.369821 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.369946 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:17:09.369924 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": container with ID starting with 3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1 not found: ID does not exist" containerID="3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1" Jul 13 20:17:09.370044 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.369948 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1"} err="failed to get container status \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": rpc error: code = NotFound desc = could not find container \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": container with ID starting with 3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1 not found: ID does not exist" Jul 13 20:17:09.370044 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.369961 2568 scope.go:117] "RemoveContainer" containerID="0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df" Jul 13 20:17:09.370222 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:17:09.370204 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": container with ID starting with 0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df not found: ID does not exist" containerID="0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df" Jul 13 20:17:09.370281 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.370228 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df"} err="failed to get container status \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": rpc error: code = NotFound desc = could not find container \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": container with ID starting with 0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df not found: ID does not exist" Jul 13 20:17:09.370281 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.370242 2568 scope.go:117] "RemoveContainer" containerID="b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2" Jul 13 20:17:09.370688 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:17:09.370642 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": container with ID starting with b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2 not found: ID does not exist" containerID="b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2" Jul 13 20:17:09.370791 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.370681 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2"} err="failed to get container status \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": rpc error: code = NotFound desc = could not find container \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": container with ID starting with b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2 not found: ID does not exist" Jul 13 20:17:09.370791 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.370702 2568 scope.go:117] "RemoveContainer" containerID="45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f" Jul 13 20:17:09.371029 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:17:09.370975 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": container with ID starting with 45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f not found: ID does not exist" containerID="45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f" Jul 13 20:17:09.371029 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.371000 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f"} err="failed to get container status \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": rpc error: code = NotFound desc = could not find container \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": container with ID starting with 45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f not found: ID does not exist" Jul 13 20:17:09.371029 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.371020 2568 scope.go:117] "RemoveContainer" containerID="83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7" Jul 13 20:17:09.371558 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.371534 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7"} err="failed to get container status \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": rpc error: code = NotFound desc = could not find container \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": container with ID starting with 83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7 not found: ID does not exist" Jul 13 20:17:09.371655 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.371560 2568 scope.go:117] "RemoveContainer" containerID="c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4" Jul 13 20:17:09.371959 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.371912 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4"} err="failed to get container status \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": rpc error: code = NotFound desc = could not find container \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": container with ID starting with c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4 not found: ID does not exist" Jul 13 20:17:09.371959 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.371937 2568 scope.go:117] "RemoveContainer" containerID="2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a" Jul 13 20:17:09.372915 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.372723 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 13 20:17:09.372915 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.372775 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-x7qfj\"" Jul 13 20:17:09.373123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.373106 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 13 20:17:09.373234 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.373213 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a"} err="failed to get container status \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": rpc error: code = NotFound desc = could not find container \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": container with ID starting with 2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a not found: ID does not exist" Jul 13 20:17:09.373334 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.373320 2568 scope.go:117] "RemoveContainer" containerID="3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1" Jul 13 20:17:09.373467 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.373450 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 13 20:17:09.373577 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.373549 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 13 20:17:09.373637 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.373385 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 13 20:17:09.373719 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.373294 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 13 20:17:09.373831 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.373815 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 13 20:17:09.373957 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.373779 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 13 20:17:09.374022 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.373995 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cs0bppg0flscg\"" Jul 13 20:17:09.374077 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.373364 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 13 20:17:09.374494 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.374468 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1"} err="failed to get container status \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": rpc error: code = NotFound desc = could not find container \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": container with ID starting with 3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1 not found: ID does not exist" Jul 13 20:17:09.374597 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.374497 2568 scope.go:117] "RemoveContainer" containerID="0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df" Jul 13 20:17:09.374833 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.374816 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 13 20:17:09.374944 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.374888 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 13 20:17:09.375156 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.375139 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 13 20:17:09.375773 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.375747 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df"} err="failed to get container status \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": rpc error: code = NotFound desc = could not find container \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": container with ID starting with 0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df not found: ID does not exist" Jul 13 20:17:09.375773 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.375774 2568 scope.go:117] "RemoveContainer" containerID="b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2" Jul 13 20:17:09.376167 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.376139 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2"} err="failed to get container status \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": rpc error: code = NotFound desc = could not find container \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": container with ID starting with b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2 not found: ID does not exist" Jul 13 20:17:09.376283 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.376170 2568 scope.go:117] "RemoveContainer" containerID="45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f" Jul 13 20:17:09.376631 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.376517 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f"} err="failed to get container status \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": rpc error: code = NotFound desc = could not find container \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": container with ID starting with 45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f not found: ID does not exist" Jul 13 20:17:09.376631 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.376545 2568 scope.go:117] "RemoveContainer" containerID="83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7" Jul 13 20:17:09.377080 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.376955 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7"} err="failed to get container status \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": rpc error: code = NotFound desc = could not find container \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": container with ID starting with 83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7 not found: ID does not exist" Jul 13 20:17:09.377080 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.376994 2568 scope.go:117] "RemoveContainer" containerID="c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4" Jul 13 20:17:09.377334 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.377297 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4"} err="failed to get container status \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": rpc error: code = NotFound desc = could not find container \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": container with ID starting with c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4 not found: ID does not exist" Jul 13 20:17:09.377450 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.377338 2568 scope.go:117] "RemoveContainer" containerID="2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a" Jul 13 20:17:09.377834 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.377814 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 13 20:17:09.377965 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.377932 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a"} err="failed to get container status \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": rpc error: code = NotFound desc = could not find container \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": container with ID starting with 2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a not found: ID does not exist" Jul 13 20:17:09.377965 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.377958 2568 scope.go:117] "RemoveContainer" containerID="3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1" Jul 13 20:17:09.378373 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.378348 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1"} err="failed to get container status \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": rpc error: code = NotFound desc = could not find container \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": container with ID starting with 3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1 not found: ID does not exist" Jul 13 20:17:09.378373 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.378373 2568 scope.go:117] "RemoveContainer" containerID="0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df" Jul 13 20:17:09.378783 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.378757 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df"} err="failed to get container status \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": rpc error: code = NotFound desc = could not find container \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": container with ID starting with 0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df not found: ID does not exist" Jul 13 20:17:09.378901 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.378784 2568 scope.go:117] "RemoveContainer" containerID="b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2" Jul 13 20:17:09.379145 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.379029 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2"} err="failed to get container status \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": rpc error: code = NotFound desc = could not find container \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": container with ID starting with b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2 not found: ID does not exist" Jul 13 20:17:09.379145 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.379055 2568 scope.go:117] "RemoveContainer" containerID="45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f" Jul 13 20:17:09.379379 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.379359 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f"} err="failed to get container status \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": rpc error: code = NotFound desc = could not find container \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": container with ID starting with 45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f not found: ID does not exist" Jul 13 20:17:09.379379 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.379379 2568 scope.go:117] "RemoveContainer" containerID="83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7" Jul 13 20:17:09.379681 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.379655 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7"} err="failed to get container status \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": rpc error: code = NotFound desc = could not find container \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": container with ID starting with 83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7 not found: ID does not exist" Jul 13 20:17:09.379782 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.379683 2568 scope.go:117] "RemoveContainer" containerID="c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4" Jul 13 20:17:09.380013 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.379987 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4"} err="failed to get container status \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": rpc error: code = NotFound desc = could not find container \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": container with ID starting with c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4 not found: ID does not exist" Jul 13 20:17:09.380013 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.380013 2568 scope.go:117] "RemoveContainer" containerID="2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a" Jul 13 20:17:09.380443 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.380403 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a"} err="failed to get container status \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": rpc error: code = NotFound desc = could not find container \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": container with ID starting with 2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a not found: ID does not exist" Jul 13 20:17:09.380545 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.380443 2568 scope.go:117] "RemoveContainer" containerID="3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1" Jul 13 20:17:09.381220 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.381200 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 13 20:17:09.381426 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.381386 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1"} err="failed to get container status \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": rpc error: code = NotFound desc = could not find container \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": container with ID starting with 3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1 not found: ID does not exist" Jul 13 20:17:09.381528 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.381430 2568 scope.go:117] "RemoveContainer" containerID="0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df" Jul 13 20:17:09.381774 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.381731 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df"} err="failed to get container status \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": rpc error: code = NotFound desc = could not find container \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": container with ID starting with 0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df not found: ID does not exist" Jul 13 20:17:09.381774 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.381754 2568 scope.go:117] "RemoveContainer" containerID="b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2" Jul 13 20:17:09.382090 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.381997 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2"} err="failed to get container status \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": rpc error: code = NotFound desc = could not find container \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": container with ID starting with b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2 not found: ID does not exist" Jul 13 20:17:09.382090 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.382020 2568 scope.go:117] "RemoveContainer" containerID="45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f" Jul 13 20:17:09.382491 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.382464 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f"} err="failed to get container status \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": rpc error: code = NotFound desc = could not find container \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": container with ID starting with 45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f not found: ID does not exist" Jul 13 20:17:09.382491 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.382491 2568 scope.go:117] "RemoveContainer" containerID="83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7" Jul 13 20:17:09.383994 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.382810 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7"} err="failed to get container status \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": rpc error: code = NotFound desc = could not find container \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": container with ID starting with 83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7 not found: ID does not exist" Jul 13 20:17:09.383994 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.382835 2568 scope.go:117] "RemoveContainer" containerID="c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4" Jul 13 20:17:09.385924 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.385393 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 20:17:09.385924 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.385536 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4"} err="failed to get container status \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": rpc error: code = NotFound desc = could not find container \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": container with ID starting with c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4 not found: ID does not exist" Jul 13 20:17:09.385924 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.385559 2568 scope.go:117] "RemoveContainer" containerID="2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a" Jul 13 20:17:09.385924 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.385844 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a"} err="failed to get container status \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": rpc error: code = NotFound desc = could not find container \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": container with ID starting with 2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a not found: ID does not exist" Jul 13 20:17:09.385924 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.385868 2568 scope.go:117] "RemoveContainer" containerID="3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1" Jul 13 20:17:09.386194 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.386104 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1"} err="failed to get container status \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": rpc error: code = NotFound desc = could not find container \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": container with ID starting with 3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1 not found: ID does not exist" Jul 13 20:17:09.386194 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.386126 2568 scope.go:117] "RemoveContainer" containerID="0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df" Jul 13 20:17:09.386537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.386434 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df"} err="failed to get container status \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": rpc error: code = NotFound desc = could not find container \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": container with ID starting with 0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df not found: ID does not exist" Jul 13 20:17:09.386537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.386459 2568 scope.go:117] "RemoveContainer" containerID="b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2" Jul 13 20:17:09.386862 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.386800 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2"} err="failed to get container status \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": rpc error: code = NotFound desc = could not find container \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": container with ID starting with b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2 not found: ID does not exist" Jul 13 20:17:09.386862 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.386828 2568 scope.go:117] "RemoveContainer" containerID="45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f" Jul 13 20:17:09.387146 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.387126 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f"} err="failed to get container status \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": rpc error: code = NotFound desc = could not find container \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": container with ID starting with 45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f not found: ID does not exist" Jul 13 20:17:09.387146 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.387147 2568 scope.go:117] "RemoveContainer" containerID="83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7" Jul 13 20:17:09.387477 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.387352 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7"} err="failed to get container status \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": rpc error: code = NotFound desc = could not find container \"83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7\": container with ID starting with 83f702c4cc9e529b9c31e94bb63f6944b6d0a9e39b28748f9c0aec1f88d64cf7 not found: ID does not exist" Jul 13 20:17:09.387477 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.387377 2568 scope.go:117] "RemoveContainer" containerID="c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4" Jul 13 20:17:09.387898 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.387643 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4"} err="failed to get container status \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": rpc error: code = NotFound desc = could not find container \"c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4\": container with ID starting with c66fa73095947981c43c43e9892292dc61f8426a6c461c3cf73e1f2e2dfcf6c4 not found: ID does not exist" Jul 13 20:17:09.387898 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.387666 2568 scope.go:117] "RemoveContainer" containerID="2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a" Jul 13 20:17:09.388452 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.388398 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a"} err="failed to get container status \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": rpc error: code = NotFound desc = could not find container \"2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a\": container with ID starting with 2e4410bc8ff21d89ff5ec2c3b93b5cbd832cbefb2bec46cc02acc65b3a01f49a not found: ID does not exist" Jul 13 20:17:09.388452 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.388438 2568 scope.go:117] "RemoveContainer" containerID="3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1" Jul 13 20:17:09.388684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.388660 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1"} err="failed to get container status \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": rpc error: code = NotFound desc = could not find container \"3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1\": container with ID starting with 3308d513c03000fe0e941af850187c4694dd4198987bd10253d86e71dfbd71c1 not found: ID does not exist" Jul 13 20:17:09.388762 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.388686 2568 scope.go:117] "RemoveContainer" containerID="0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df" Jul 13 20:17:09.389109 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.389081 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df"} err="failed to get container status \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": rpc error: code = NotFound desc = could not find container \"0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df\": container with ID starting with 0da651d84e1ba8f475473daf83cee926e76c9841ad47188da6e767ac5afcb7df not found: ID does not exist" Jul 13 20:17:09.389198 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.389111 2568 scope.go:117] "RemoveContainer" containerID="b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2" Jul 13 20:17:09.389516 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.389398 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2"} err="failed to get container status \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": rpc error: code = NotFound desc = could not find container \"b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2\": container with ID starting with b34caa21b13a792c75298348ccfb69a744880ebe80d2549936d08ee6804a67d2 not found: ID does not exist" Jul 13 20:17:09.389516 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.389446 2568 scope.go:117] "RemoveContainer" containerID="45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f" Jul 13 20:17:09.389893 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.389869 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f"} err="failed to get container status \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": rpc error: code = NotFound desc = could not find container \"45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f\": container with ID starting with 45cbeb42e827dba22a61eee84f0cb069f64f29f85204a3862377f1d5d925633f not found: ID does not exist" Jul 13 20:17:09.403978 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.403951 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c57bed8-a984-4b69-8f03-98f076b8bb4f" path="/var/lib/kubelet/pods/4c57bed8-a984-4b69-8f03-98f076b8bb4f/volumes" Jul 13 20:17:09.451317 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.451537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451364 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.451537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451431 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.451537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451462 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.451704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451613 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.451751 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451707 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-config-out\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.451751 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451737 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.451829 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451764 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.451829 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451799 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.451909 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451866 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k8dq\" (UniqueName: \"kubernetes.io/projected/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-kube-api-access-7k8dq\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.451960 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451943 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.451998 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.451980 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.452043 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.452007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.452043 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.452032 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-web-config\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.452120 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.452054 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.452152 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.452111 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.452219 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.452202 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-config\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.452283 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.452267 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.452343 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.452299 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.452343 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.452329 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.553283 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553246 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.553495 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553328 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.553495 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553363 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-config-out\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.553495 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553390 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.553703 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553584 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.553703 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553640 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.553703 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553673 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k8dq\" (UniqueName: \"kubernetes.io/projected/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-kube-api-access-7k8dq\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.553866 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553707 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.553866 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553766 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.553866 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553804 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.553866 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-web-config\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.554052 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553873 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.554052 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.554052 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553972 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-config\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.554052 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.553998 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.554052 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.554025 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.554314 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.554057 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.554314 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.554097 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.554314 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.554150 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.554314 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.554196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.554551 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.554537 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.556618 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.554974 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.556618 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.554054 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.556618 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.555742 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.556618 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.556230 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.556907 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.556669 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.557338 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.557084 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.557338 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.557188 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.557338 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.557298 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.559094 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.559052 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-config-out\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.559295 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.559261 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.559295 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.559283 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.559748 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.559649 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.560068 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.560000 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-web-config\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.560360 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.560339 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.560548 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.560528 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.560598 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.560551 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.560643 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.560615 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.561087 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.561065 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-config\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.563382 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.563363 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k8dq\" (UniqueName: \"kubernetes.io/projected/1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1-kube-api-access-7k8dq\") pod \"prometheus-k8s-0\" (UID: \"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.686586 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.686554 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:09.842967 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:09.842944 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 13 20:17:09.844875 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:17:09.844847 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa7f0f6_7f77_4086_ae6d_5d7af9bdf1b1.slice/crio-bef534abefa1f3991a8d80078bdbc7d3e7c5515e2329cad7183b6710d95759e2 WatchSource:0}: Error finding container bef534abefa1f3991a8d80078bdbc7d3e7c5515e2329cad7183b6710d95759e2: Status 404 returned error can't find the container with id bef534abefa1f3991a8d80078bdbc7d3e7c5515e2329cad7183b6710d95759e2 Jul 13 20:17:10.312182 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:10.312150 2568 generic.go:356] "Generic (PLEG): container finished" podID="1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1" containerID="40f4c15e34454eb8f778f316f140ca83447fc2b739c59b3b9810fee0069f44f1" exitCode=0 Jul 13 20:17:10.312578 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:10.312213 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1","Type":"ContainerDied","Data":"40f4c15e34454eb8f778f316f140ca83447fc2b739c59b3b9810fee0069f44f1"} Jul 13 20:17:10.312578 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:10.312293 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1","Type":"ContainerStarted","Data":"bef534abefa1f3991a8d80078bdbc7d3e7c5515e2329cad7183b6710d95759e2"} Jul 13 20:17:11.318883 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.318846 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" event={"ID":"0c2f3ae3-90de-403b-b7fd-050f4ec77f47","Type":"ContainerStarted","Data":"6b020250377f3d188aabce31c470d03d81c821ab3c76291e3162a8baef9f65f6"} Jul 13 20:17:11.319328 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.318915 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" event={"ID":"0c2f3ae3-90de-403b-b7fd-050f4ec77f47","Type":"ContainerStarted","Data":"6de50190ff9e6579254cf93e20be5dcc97a3507afd4b784d06a631bd96f333ef"} Jul 13 20:17:11.319328 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.318946 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" event={"ID":"0c2f3ae3-90de-403b-b7fd-050f4ec77f47","Type":"ContainerStarted","Data":"f3f1c222961492687eb256f280fbd73d743eb05902e72e92799fe49ac06936e9"} Jul 13 20:17:11.322078 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.322051 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1","Type":"ContainerStarted","Data":"7e50fac153f047a687349c533b1346af748d988c56b66c3c2bc6ac4330a679d2"} Jul 13 20:17:11.322201 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.322117 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1","Type":"ContainerStarted","Data":"f078707855eb50207993443fd9c766fa1c2c62b2d4a22da73d3a5f2b5d4e3783"} Jul 13 20:17:11.322201 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.322157 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1","Type":"ContainerStarted","Data":"c4240156d70843cdac53d8bb2dd27b4ee35ed1bb81006db440533506627f7ba1"} Jul 13 20:17:11.322343 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.322203 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1","Type":"ContainerStarted","Data":"07c2c26ecfa98d6ba9cea597a349790227f0fc03e9901d4534d696a7e82fb93b"} Jul 13 20:17:11.322343 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.322257 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1","Type":"ContainerStarted","Data":"9e9a3a5a8590c3cbd79ba3631ba62f41ccd87c0496723a0149b5f7e849b8f9f8"} Jul 13 20:17:11.322473 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.322331 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1","Type":"ContainerStarted","Data":"c9b5185545a378a2237dd591f3d4802655bbf6b008b6c070ce395b11f218cb43"} Jul 13 20:17:11.341262 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.341212 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b547fd5f-mmcvv" podStartSLOduration=1.064452306 podStartE2EDuration="3.341197072s" totalImagesPullingTime="2.276744766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:17:11.339297069 +0000 UTC m=+162.485564234" watchObservedRunningTime="2026-07-13 20:17:11.341197072 +0000 UTC m=+162.487464232" Jul 13 20:17:11.367704 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.367644 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.34667544 podStartE2EDuration="2.367623747s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.948307ms" isStatefulPod=false podCreationTimestamp="2026-07-13 20:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:17:11.366283276 +0000 UTC m=+162.512550441" watchObservedRunningTime="2026-07-13 20:17:11.367623747 +0000 UTC m=+162.513890909" Jul 13 20:17:11.958514 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.958478 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55c9bc7d64-pjpq9"] Jul 13 20:17:11.961483 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.961462 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:11.975762 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:11.975735 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c9bc7d64-pjpq9"] Jul 13 20:17:12.079477 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.079441 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-oauth-config\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.079477 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.079492 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-service-ca\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.079734 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.079564 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-oauth-serving-cert\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.079734 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.079605 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-console-config\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.079734 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.079627 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-trusted-ca-bundle\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.079734 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.079694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tzgg\" (UniqueName: \"kubernetes.io/projected/b55cb24b-1b07-4013-8da7-987792ece294-kube-api-access-4tzgg\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.079734 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.079712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-serving-cert\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.180948 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.180912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-oauth-config\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.181094 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.180960 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-service-ca\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.181094 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.181007 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-oauth-serving-cert\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.181094 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.181034 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-console-config\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.181094 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.181061 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-trusted-ca-bundle\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.181315 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.181107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tzgg\" (UniqueName: \"kubernetes.io/projected/b55cb24b-1b07-4013-8da7-987792ece294-kube-api-access-4tzgg\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.181315 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.181135 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-serving-cert\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.181898 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.181864 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-oauth-serving-cert\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.182028 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.181913 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-console-config\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.182028 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.182019 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-service-ca\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.182669 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.182643 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-trusted-ca-bundle\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.183537 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.183512 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-oauth-config\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.183722 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.183704 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-serving-cert\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.189406 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.189385 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tzgg\" (UniqueName: \"kubernetes.io/projected/b55cb24b-1b07-4013-8da7-987792ece294-kube-api-access-4tzgg\") pod \"console-55c9bc7d64-pjpq9\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.270811 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.270723 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:12.419434 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:12.419387 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c9bc7d64-pjpq9"] Jul 13 20:17:12.421655 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:17:12.421626 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55cb24b_1b07_4013_8da7_987792ece294.slice/crio-7a616320daca4c09d819e855773dd9501855634c11cc3ad663434e87943b58a1 WatchSource:0}: Error finding container 7a616320daca4c09d819e855773dd9501855634c11cc3ad663434e87943b58a1: Status 404 returned error can't find the container with id 7a616320daca4c09d819e855773dd9501855634c11cc3ad663434e87943b58a1 Jul 13 20:17:13.337876 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:13.337838 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c9bc7d64-pjpq9" event={"ID":"b55cb24b-1b07-4013-8da7-987792ece294","Type":"ContainerStarted","Data":"447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe"} Jul 13 20:17:13.338051 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:13.337892 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c9bc7d64-pjpq9" event={"ID":"b55cb24b-1b07-4013-8da7-987792ece294","Type":"ContainerStarted","Data":"7a616320daca4c09d819e855773dd9501855634c11cc3ad663434e87943b58a1"} Jul 13 20:17:13.356429 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:13.356377 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55c9bc7d64-pjpq9" podStartSLOduration=2.356362166 podStartE2EDuration="2.356362166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:17:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:17:13.354863978 +0000 UTC m=+164.501131142" watchObservedRunningTime="2026-07-13 20:17:13.356362166 +0000 UTC m=+164.502629327" Jul 13 20:17:14.687652 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:14.687620 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:17:22.271070 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:22.270988 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:22.271070 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:22.271059 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:22.275602 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:22.275577 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:22.383625 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:22.383598 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:17:22.434994 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:22.434964 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f7fc494c5-gck5b"] Jul 13 20:17:47.455067 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.455025 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f7fc494c5-gck5b" podUID="367c8878-77ed-4d3d-9105-23aef80fda79" containerName="console" containerID="cri-o://f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5" gracePeriod=15 Jul 13 20:17:47.708988 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.708936 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:17:47.804478 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.804448 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-oauth-serving-cert\") pod \"367c8878-77ed-4d3d-9105-23aef80fda79\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " Jul 13 20:17:47.804615 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.804523 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-trusted-ca-bundle\") pod \"367c8878-77ed-4d3d-9105-23aef80fda79\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " Jul 13 20:17:47.804615 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.804593 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-serving-cert\") pod \"367c8878-77ed-4d3d-9105-23aef80fda79\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " Jul 13 20:17:47.804777 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.804757 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-service-ca\") pod \"367c8878-77ed-4d3d-9105-23aef80fda79\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " Jul 13 20:17:47.804835 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.804808 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-oauth-config\") pod \"367c8878-77ed-4d3d-9105-23aef80fda79\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " Jul 13 20:17:47.804883 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.804831 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-console-config\") pod \"367c8878-77ed-4d3d-9105-23aef80fda79\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " Jul 13 20:17:47.804883 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.804835 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "367c8878-77ed-4d3d-9105-23aef80fda79" (UID: "367c8878-77ed-4d3d-9105-23aef80fda79"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:47.804883 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.804868 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjqbj\" (UniqueName: \"kubernetes.io/projected/367c8878-77ed-4d3d-9105-23aef80fda79-kube-api-access-kjqbj\") pod \"367c8878-77ed-4d3d-9105-23aef80fda79\" (UID: \"367c8878-77ed-4d3d-9105-23aef80fda79\") " Jul 13 20:17:47.805030 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.804884 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "367c8878-77ed-4d3d-9105-23aef80fda79" (UID: "367c8878-77ed-4d3d-9105-23aef80fda79"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:47.805124 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.805099 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-service-ca" (OuterVolumeSpecName: "service-ca") pod "367c8878-77ed-4d3d-9105-23aef80fda79" (UID: "367c8878-77ed-4d3d-9105-23aef80fda79"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:47.805211 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.805195 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-oauth-serving-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:47.805268 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.805220 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-trusted-ca-bundle\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:47.805268 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.805228 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-console-config" (OuterVolumeSpecName: "console-config") pod "367c8878-77ed-4d3d-9105-23aef80fda79" (UID: "367c8878-77ed-4d3d-9105-23aef80fda79"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:17:47.805268 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.805240 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:47.806815 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.806782 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "367c8878-77ed-4d3d-9105-23aef80fda79" (UID: "367c8878-77ed-4d3d-9105-23aef80fda79"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:47.806815 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.806797 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "367c8878-77ed-4d3d-9105-23aef80fda79" (UID: "367c8878-77ed-4d3d-9105-23aef80fda79"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:17:47.806988 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.806839 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/367c8878-77ed-4d3d-9105-23aef80fda79-kube-api-access-kjqbj" (OuterVolumeSpecName: "kube-api-access-kjqbj") pod "367c8878-77ed-4d3d-9105-23aef80fda79" (UID: "367c8878-77ed-4d3d-9105-23aef80fda79"). InnerVolumeSpecName "kube-api-access-kjqbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:17:47.905676 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.905643 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-serving-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:47.905676 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.905669 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/367c8878-77ed-4d3d-9105-23aef80fda79-console-oauth-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:47.905676 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.905678 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/367c8878-77ed-4d3d-9105-23aef80fda79-console-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:47.905887 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:47.905687 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjqbj\" (UniqueName: \"kubernetes.io/projected/367c8878-77ed-4d3d-9105-23aef80fda79-kube-api-access-kjqbj\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:17:48.490980 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:48.490944 2568 generic.go:356] "Generic (PLEG): container finished" podID="367c8878-77ed-4d3d-9105-23aef80fda79" containerID="f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5" exitCode=0 Jul 13 20:17:48.491396 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:48.490989 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f7fc494c5-gck5b" Jul 13 20:17:48.491396 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:48.491027 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f7fc494c5-gck5b" event={"ID":"367c8878-77ed-4d3d-9105-23aef80fda79","Type":"ContainerDied","Data":"f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5"} Jul 13 20:17:48.491396 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:48.491079 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f7fc494c5-gck5b" event={"ID":"367c8878-77ed-4d3d-9105-23aef80fda79","Type":"ContainerDied","Data":"066935be03b340bc3aa94e55b0d844280ec3e18d496cb1e722d0bd9ee7d1a133"} Jul 13 20:17:48.491396 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:48.491100 2568 scope.go:117] "RemoveContainer" containerID="f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5" Jul 13 20:17:48.500758 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:48.500737 2568 scope.go:117] "RemoveContainer" containerID="f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5" Jul 13 20:17:48.501003 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:17:48.500985 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5\": container with ID starting with f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5 not found: ID does not exist" containerID="f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5" Jul 13 20:17:48.501050 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:48.501012 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5"} err="failed to get container status \"f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5\": rpc error: code = NotFound desc = could not find container \"f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5\": container with ID starting with f1fe9beb16906b7c9ad47639506f27ddafdc1ae808b79e898411652c518a5cb5 not found: ID does not exist" Jul 13 20:17:48.513216 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:48.513188 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f7fc494c5-gck5b"] Jul 13 20:17:48.516658 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:48.516639 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f7fc494c5-gck5b"] Jul 13 20:17:49.403182 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:17:49.403147 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="367c8878-77ed-4d3d-9105-23aef80fda79" path="/var/lib/kubelet/pods/367c8878-77ed-4d3d-9105-23aef80fda79/volumes" Jul 13 20:18:09.687602 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:09.687565 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:18:09.702976 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:09.702951 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:18:10.595535 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:10.595510 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 13 20:18:27.395335 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.395302 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57fb674c9b-qhtzk"] Jul 13 20:18:27.395819 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.395684 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="367c8878-77ed-4d3d-9105-23aef80fda79" containerName="console" Jul 13 20:18:27.395819 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.395697 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="367c8878-77ed-4d3d-9105-23aef80fda79" containerName="console" Jul 13 20:18:27.395819 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.395756 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="367c8878-77ed-4d3d-9105-23aef80fda79" containerName="console" Jul 13 20:18:27.398900 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.398870 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.409626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.409604 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57fb674c9b-qhtzk"] Jul 13 20:18:27.451392 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.451353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-trusted-ca-bundle\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.451608 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.451436 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-oauth-config\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.451608 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.451506 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-service-ca\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.451608 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.451559 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-config\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.451608 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.451578 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lngjx\" (UniqueName: \"kubernetes.io/projected/bb7a259a-fe9f-4e87-8c75-6d179bffe340-kube-api-access-lngjx\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.451608 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.451599 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-oauth-serving-cert\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.451907 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.451636 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-serving-cert\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.552597 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.552562 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-trusted-ca-bundle\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.552597 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.552604 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-oauth-config\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.552830 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.552623 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-service-ca\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.552830 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.552642 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-config\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.552830 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.552658 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lngjx\" (UniqueName: \"kubernetes.io/projected/bb7a259a-fe9f-4e87-8c75-6d179bffe340-kube-api-access-lngjx\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.552830 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.552677 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-oauth-serving-cert\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.552830 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.552713 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-serving-cert\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.559429 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.553816 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-config\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.559429 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.553912 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-service-ca\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.559429 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.553981 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-oauth-serving-cert\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.559429 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.555470 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-trusted-ca-bundle\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.559862 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.559820 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-oauth-config\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.560222 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.560205 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-serving-cert\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.563958 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.563937 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lngjx\" (UniqueName: \"kubernetes.io/projected/bb7a259a-fe9f-4e87-8c75-6d179bffe340-kube-api-access-lngjx\") pod \"console-57fb674c9b-qhtzk\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.708634 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.708542 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:27.837311 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:27.837283 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57fb674c9b-qhtzk"] Jul 13 20:18:27.839570 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:18:27.839542 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb7a259a_fe9f_4e87_8c75_6d179bffe340.slice/crio-e4e2d1316ecfd2ceec9ffa91f5a79d90128e6c34b2762b937db79be893869c1c WatchSource:0}: Error finding container e4e2d1316ecfd2ceec9ffa91f5a79d90128e6c34b2762b937db79be893869c1c: Status 404 returned error can't find the container with id e4e2d1316ecfd2ceec9ffa91f5a79d90128e6c34b2762b937db79be893869c1c Jul 13 20:18:28.654375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:28.654336 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fb674c9b-qhtzk" event={"ID":"bb7a259a-fe9f-4e87-8c75-6d179bffe340","Type":"ContainerStarted","Data":"04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9"} Jul 13 20:18:28.654749 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:28.654402 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fb674c9b-qhtzk" event={"ID":"bb7a259a-fe9f-4e87-8c75-6d179bffe340","Type":"ContainerStarted","Data":"e4e2d1316ecfd2ceec9ffa91f5a79d90128e6c34b2762b937db79be893869c1c"} Jul 13 20:18:28.677876 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:28.677832 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57fb674c9b-qhtzk" podStartSLOduration=1.67781667 podStartE2EDuration="1.67781667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:18:28.676010425 +0000 UTC m=+239.822277586" watchObservedRunningTime="2026-07-13 20:18:28.67781667 +0000 UTC m=+239.824083832" Jul 13 20:18:37.709495 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:37.709460 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:37.709866 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:37.709530 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:37.716626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:37.716150 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:38.697710 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:38.697680 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:18:38.746119 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:18:38.746088 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c9bc7d64-pjpq9"] Jul 13 20:19:03.766876 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:03.766764 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55c9bc7d64-pjpq9" podUID="b55cb24b-1b07-4013-8da7-987792ece294" containerName="console" containerID="cri-o://447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe" gracePeriod=15 Jul 13 20:19:04.009137 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.009112 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:19:04.082036 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.082004 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-serving-cert\") pod \"b55cb24b-1b07-4013-8da7-987792ece294\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " Jul 13 20:19:04.082203 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.082074 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-service-ca\") pod \"b55cb24b-1b07-4013-8da7-987792ece294\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " Jul 13 20:19:04.082203 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.082099 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-trusted-ca-bundle\") pod \"b55cb24b-1b07-4013-8da7-987792ece294\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " Jul 13 20:19:04.082203 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.082120 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-console-config\") pod \"b55cb24b-1b07-4013-8da7-987792ece294\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " Jul 13 20:19:04.082203 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.082159 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-oauth-serving-cert\") pod \"b55cb24b-1b07-4013-8da7-987792ece294\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " Jul 13 20:19:04.082473 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.082204 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-oauth-config\") pod \"b55cb24b-1b07-4013-8da7-987792ece294\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " Jul 13 20:19:04.082473 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.082250 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tzgg\" (UniqueName: \"kubernetes.io/projected/b55cb24b-1b07-4013-8da7-987792ece294-kube-api-access-4tzgg\") pod \"b55cb24b-1b07-4013-8da7-987792ece294\" (UID: \"b55cb24b-1b07-4013-8da7-987792ece294\") " Jul 13 20:19:04.082664 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.082628 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b55cb24b-1b07-4013-8da7-987792ece294" (UID: "b55cb24b-1b07-4013-8da7-987792ece294"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:19:04.082664 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.082640 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-service-ca" (OuterVolumeSpecName: "service-ca") pod "b55cb24b-1b07-4013-8da7-987792ece294" (UID: "b55cb24b-1b07-4013-8da7-987792ece294"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:19:04.082664 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.082645 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-console-config" (OuterVolumeSpecName: "console-config") pod "b55cb24b-1b07-4013-8da7-987792ece294" (UID: "b55cb24b-1b07-4013-8da7-987792ece294"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:19:04.082791 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.082675 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b55cb24b-1b07-4013-8da7-987792ece294" (UID: "b55cb24b-1b07-4013-8da7-987792ece294"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:19:04.084296 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.084261 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b55cb24b-1b07-4013-8da7-987792ece294" (UID: "b55cb24b-1b07-4013-8da7-987792ece294"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:19:04.084397 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.084304 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b55cb24b-1b07-4013-8da7-987792ece294" (UID: "b55cb24b-1b07-4013-8da7-987792ece294"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:19:04.084397 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.084367 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b55cb24b-1b07-4013-8da7-987792ece294-kube-api-access-4tzgg" (OuterVolumeSpecName: "kube-api-access-4tzgg") pod "b55cb24b-1b07-4013-8da7-987792ece294" (UID: "b55cb24b-1b07-4013-8da7-987792ece294"). InnerVolumeSpecName "kube-api-access-4tzgg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:19:04.183196 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.183166 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:19:04.183196 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.183192 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-trusted-ca-bundle\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:19:04.183196 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.183202 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-console-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:19:04.183450 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.183211 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b55cb24b-1b07-4013-8da7-987792ece294-oauth-serving-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:19:04.183450 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.183221 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-oauth-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:19:04.183450 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.183231 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tzgg\" (UniqueName: \"kubernetes.io/projected/b55cb24b-1b07-4013-8da7-987792ece294-kube-api-access-4tzgg\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:19:04.183450 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.183240 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b55cb24b-1b07-4013-8da7-987792ece294-console-serving-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:19:04.811489 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.811452 2568 generic.go:356] "Generic (PLEG): container finished" podID="b55cb24b-1b07-4013-8da7-987792ece294" containerID="447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe" exitCode=0 Jul 13 20:19:04.811897 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.811502 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c9bc7d64-pjpq9" Jul 13 20:19:04.811897 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.811538 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c9bc7d64-pjpq9" event={"ID":"b55cb24b-1b07-4013-8da7-987792ece294","Type":"ContainerDied","Data":"447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe"} Jul 13 20:19:04.811897 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.811591 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c9bc7d64-pjpq9" event={"ID":"b55cb24b-1b07-4013-8da7-987792ece294","Type":"ContainerDied","Data":"7a616320daca4c09d819e855773dd9501855634c11cc3ad663434e87943b58a1"} Jul 13 20:19:04.811897 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.811617 2568 scope.go:117] "RemoveContainer" containerID="447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe" Jul 13 20:19:04.823352 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.823333 2568 scope.go:117] "RemoveContainer" containerID="447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe" Jul 13 20:19:04.823615 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:19:04.823598 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe\": container with ID starting with 447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe not found: ID does not exist" containerID="447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe" Jul 13 20:19:04.823663 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.823624 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe"} err="failed to get container status \"447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe\": rpc error: code = NotFound desc = could not find container \"447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe\": container with ID starting with 447d15d38bc4c27e67035c8f780673e6e739fc62c60b84ac6134eac70d6235fe not found: ID does not exist" Jul 13 20:19:04.834520 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.834498 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c9bc7d64-pjpq9"] Jul 13 20:19:04.837281 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:04.837260 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55c9bc7d64-pjpq9"] Jul 13 20:19:05.401728 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:05.401693 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b55cb24b-1b07-4013-8da7-987792ece294" path="/var/lib/kubelet/pods/b55cb24b-1b07-4013-8da7-987792ece294/volumes" Jul 13 20:19:29.261358 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:29.261333 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:19:29.264627 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:29.264604 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:19:29.266527 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:29.266508 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 20:19:49.598747 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.598714 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc"] Jul 13 20:19:49.601189 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.599031 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b55cb24b-1b07-4013-8da7-987792ece294" containerName="console" Jul 13 20:19:49.601189 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.599041 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b55cb24b-1b07-4013-8da7-987792ece294" containerName="console" Jul 13 20:19:49.601189 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.599104 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b55cb24b-1b07-4013-8da7-987792ece294" containerName="console" Jul 13 20:19:49.601997 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.601982 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc" Jul 13 20:19:49.604448 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.604428 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 20:19:49.605181 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.605166 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ndqjq\"" Jul 13 20:19:49.605289 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.605190 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 20:19:49.611012 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.610992 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc"] Jul 13 20:19:49.682517 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.682489 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db32c3f2-1f66-4ed9-b42d-6910a1d0e16f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kd8jc\" (UID: \"db32c3f2-1f66-4ed9-b42d-6910a1d0e16f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc" Jul 13 20:19:49.682643 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.682543 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8bqk\" (UniqueName: \"kubernetes.io/projected/db32c3f2-1f66-4ed9-b42d-6910a1d0e16f-kube-api-access-j8bqk\") pod \"cert-manager-cainjector-559cdd8c87-kd8jc\" (UID: \"db32c3f2-1f66-4ed9-b42d-6910a1d0e16f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc" Jul 13 20:19:49.783894 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.783862 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8bqk\" (UniqueName: \"kubernetes.io/projected/db32c3f2-1f66-4ed9-b42d-6910a1d0e16f-kube-api-access-j8bqk\") pod \"cert-manager-cainjector-559cdd8c87-kd8jc\" (UID: \"db32c3f2-1f66-4ed9-b42d-6910a1d0e16f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc" Jul 13 20:19:49.784050 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.783934 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db32c3f2-1f66-4ed9-b42d-6910a1d0e16f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kd8jc\" (UID: \"db32c3f2-1f66-4ed9-b42d-6910a1d0e16f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc" Jul 13 20:19:49.792268 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.792246 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db32c3f2-1f66-4ed9-b42d-6910a1d0e16f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kd8jc\" (UID: \"db32c3f2-1f66-4ed9-b42d-6910a1d0e16f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc" Jul 13 20:19:49.792389 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.792296 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8bqk\" (UniqueName: \"kubernetes.io/projected/db32c3f2-1f66-4ed9-b42d-6910a1d0e16f-kube-api-access-j8bqk\") pod \"cert-manager-cainjector-559cdd8c87-kd8jc\" (UID: \"db32c3f2-1f66-4ed9-b42d-6910a1d0e16f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc" Jul 13 20:19:49.926604 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:49.926516 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc" Jul 13 20:19:50.056378 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:50.056350 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc"] Jul 13 20:19:50.059890 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:19:50.059862 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb32c3f2_1f66_4ed9_b42d_6910a1d0e16f.slice/crio-f4380399a066af2ab728bc2ef450586894514d60f28407dcc7cef065d95a9bce WatchSource:0}: Error finding container f4380399a066af2ab728bc2ef450586894514d60f28407dcc7cef065d95a9bce: Status 404 returned error can't find the container with id f4380399a066af2ab728bc2ef450586894514d60f28407dcc7cef065d95a9bce Jul 13 20:19:50.061686 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:50.061667 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:19:51.001512 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:51.001475 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc" event={"ID":"db32c3f2-1f66-4ed9-b42d-6910a1d0e16f","Type":"ContainerStarted","Data":"f4380399a066af2ab728bc2ef450586894514d60f28407dcc7cef065d95a9bce"} Jul 13 20:19:53.012561 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:53.012510 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc" event={"ID":"db32c3f2-1f66-4ed9-b42d-6910a1d0e16f","Type":"ContainerStarted","Data":"50420f8978f1606605fb01a87171d0fc70af84f57ca164b87cb7c9ec04929509"} Jul 13 20:19:53.037440 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:53.033003 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kd8jc" podStartSLOduration=1.27705718 podStartE2EDuration="4.032983499s" totalImagesPullingTime="2.755926319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:53.031389141 +0000 UTC m=+324.177656306" watchObservedRunningTime="2026-07-13 20:19:53.032983499 +0000 UTC m=+324.179250663" Jul 13 20:19:59.479689 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.479656 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5"] Jul 13 20:19:59.483054 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.483038 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5" Jul 13 20:19:59.485471 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.485452 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:19:59.486408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.486388 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-g9g8g\"" Jul 13 20:19:59.486408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.486425 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 20:19:59.492149 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.492129 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5"] Jul 13 20:19:59.567375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.567342 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae7eca11-d4f9-4e8b-927b-d76de1854f0c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vfbx5\" (UID: \"ae7eca11-d4f9-4e8b-927b-d76de1854f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5" Jul 13 20:19:59.567375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.567379 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5qzq\" (UniqueName: \"kubernetes.io/projected/ae7eca11-d4f9-4e8b-927b-d76de1854f0c-kube-api-access-s5qzq\") pod \"openshift-lws-operator-fd8ccff4c-vfbx5\" (UID: \"ae7eca11-d4f9-4e8b-927b-d76de1854f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5" Jul 13 20:19:59.667941 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.667902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae7eca11-d4f9-4e8b-927b-d76de1854f0c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vfbx5\" (UID: \"ae7eca11-d4f9-4e8b-927b-d76de1854f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5" Jul 13 20:19:59.667941 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.667941 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5qzq\" (UniqueName: \"kubernetes.io/projected/ae7eca11-d4f9-4e8b-927b-d76de1854f0c-kube-api-access-s5qzq\") pod \"openshift-lws-operator-fd8ccff4c-vfbx5\" (UID: \"ae7eca11-d4f9-4e8b-927b-d76de1854f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5" Jul 13 20:19:59.668296 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.668275 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae7eca11-d4f9-4e8b-927b-d76de1854f0c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vfbx5\" (UID: \"ae7eca11-d4f9-4e8b-927b-d76de1854f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5" Jul 13 20:19:59.676862 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.676835 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5qzq\" (UniqueName: \"kubernetes.io/projected/ae7eca11-d4f9-4e8b-927b-d76de1854f0c-kube-api-access-s5qzq\") pod \"openshift-lws-operator-fd8ccff4c-vfbx5\" (UID: \"ae7eca11-d4f9-4e8b-927b-d76de1854f0c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5" Jul 13 20:19:59.793164 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.793141 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5" Jul 13 20:19:59.919233 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:19:59.919207 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5"] Jul 13 20:19:59.922149 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:19:59.922125 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae7eca11_d4f9_4e8b_927b_d76de1854f0c.slice/crio-989d75fc8385c34b66fa1a7704169a92ed06a597f637d1e8e9ffc9bdea546cbb WatchSource:0}: Error finding container 989d75fc8385c34b66fa1a7704169a92ed06a597f637d1e8e9ffc9bdea546cbb: Status 404 returned error can't find the container with id 989d75fc8385c34b66fa1a7704169a92ed06a597f637d1e8e9ffc9bdea546cbb Jul 13 20:20:00.042430 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:00.042390 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5" event={"ID":"ae7eca11-d4f9-4e8b-927b-d76de1854f0c","Type":"ContainerStarted","Data":"989d75fc8385c34b66fa1a7704169a92ed06a597f637d1e8e9ffc9bdea546cbb"} Jul 13 20:20:03.059055 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:03.059021 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5" event={"ID":"ae7eca11-d4f9-4e8b-927b-d76de1854f0c","Type":"ContainerStarted","Data":"95ca3e7c90fa35ccbaa179ce6d7f6eba77f96530696b57d7c42b533e4b97e667"} Jul 13 20:20:03.077725 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:03.077661 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vfbx5" podStartSLOduration=1.248533073 podStartE2EDuration="4.077642811s" totalImagesPullingTime="2.829109738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:03.077035406 +0000 UTC m=+334.223302571" watchObservedRunningTime="2026-07-13 20:20:03.077642811 +0000 UTC m=+334.223909974" Jul 13 20:20:23.390743 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.390664 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5"] Jul 13 20:20:23.397086 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.397060 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.399992 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.399969 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 20:20:23.400120 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.400022 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 20:20:23.400120 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.400053 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 20:20:23.400338 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.400322 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n4gfr\"" Jul 13 20:20:23.400338 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.400330 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 20:20:23.411443 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.411394 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5"] Jul 13 20:20:23.486006 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.485973 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85f52de8-87a0-42fc-a855-dbd5d0282791-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.486218 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.486038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85f52de8-87a0-42fc-a855-dbd5d0282791-manifests\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.486218 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.486114 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85f52de8-87a0-42fc-a855-dbd5d0282791-tmp\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.486218 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.486184 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85f52de8-87a0-42fc-a855-dbd5d0282791-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.486218 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.486216 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqncf\" (UniqueName: \"kubernetes.io/projected/85f52de8-87a0-42fc-a855-dbd5d0282791-kube-api-access-pqncf\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.587500 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.587464 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85f52de8-87a0-42fc-a855-dbd5d0282791-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.587696 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.587537 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85f52de8-87a0-42fc-a855-dbd5d0282791-manifests\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.587696 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.587568 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85f52de8-87a0-42fc-a855-dbd5d0282791-tmp\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.587696 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.587593 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85f52de8-87a0-42fc-a855-dbd5d0282791-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.587696 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.587610 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqncf\" (UniqueName: \"kubernetes.io/projected/85f52de8-87a0-42fc-a855-dbd5d0282791-kube-api-access-pqncf\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.588057 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.588031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85f52de8-87a0-42fc-a855-dbd5d0282791-manifests\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.589979 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.589957 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85f52de8-87a0-42fc-a855-dbd5d0282791-tmp\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.590077 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.590035 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85f52de8-87a0-42fc-a855-dbd5d0282791-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.590077 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.590059 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85f52de8-87a0-42fc-a855-dbd5d0282791-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.596222 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.596203 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqncf\" (UniqueName: \"kubernetes.io/projected/85f52de8-87a0-42fc-a855-dbd5d0282791-kube-api-access-pqncf\") pod \"opendatahub-operator-controller-manager-7b56dd55f-p7gf5\" (UID: \"85f52de8-87a0-42fc-a855-dbd5d0282791\") " pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.708166 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.708079 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:23.844045 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:23.844008 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5"] Jul 13 20:20:23.848088 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:20:23.848054 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85f52de8_87a0_42fc_a855_dbd5d0282791.slice/crio-7283cc015d8a1cd4aa4ccc824b4a5a73303166d4930adaad28ca7fa2126144c3 WatchSource:0}: Error finding container 7283cc015d8a1cd4aa4ccc824b4a5a73303166d4930adaad28ca7fa2126144c3: Status 404 returned error can't find the container with id 7283cc015d8a1cd4aa4ccc824b4a5a73303166d4930adaad28ca7fa2126144c3 Jul 13 20:20:24.151406 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:24.151371 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" event={"ID":"85f52de8-87a0-42fc-a855-dbd5d0282791","Type":"ContainerStarted","Data":"7283cc015d8a1cd4aa4ccc824b4a5a73303166d4930adaad28ca7fa2126144c3"} Jul 13 20:20:28.173320 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:28.173287 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" event={"ID":"85f52de8-87a0-42fc-a855-dbd5d0282791","Type":"ContainerStarted","Data":"d1604f6a97a600a8da76ba7648b661f5917e77ddd8eb5d346a3fbf2720270320"} Jul 13 20:20:29.179485 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:29.179451 2568 generic.go:356] "Generic (PLEG): container finished" podID="85f52de8-87a0-42fc-a855-dbd5d0282791" containerID="d1604f6a97a600a8da76ba7648b661f5917e77ddd8eb5d346a3fbf2720270320" exitCode=0 Jul 13 20:20:29.179856 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:29.179515 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" event={"ID":"85f52de8-87a0-42fc-a855-dbd5d0282791","Type":"ContainerDied","Data":"d1604f6a97a600a8da76ba7648b661f5917e77ddd8eb5d346a3fbf2720270320"} Jul 13 20:20:30.186340 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:30.186295 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" event={"ID":"85f52de8-87a0-42fc-a855-dbd5d0282791","Type":"ContainerStarted","Data":"96ccd0d80411c52eee58b02651d5937a6e5d487710527aa2eb4ee6851cf03bf0"} Jul 13 20:20:30.186825 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:30.186392 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:30.213770 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:30.213725 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" podStartSLOduration=2.128415671 podStartE2EDuration="7.21370924s" totalImagesPullingTime="4.034450467s" totalInitContainerRuntime="1.050843102s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:30.211158559 +0000 UTC m=+361.357425721" watchObservedRunningTime="2026-07-13 20:20:30.21370924 +0000 UTC m=+361.359976400" Jul 13 20:20:41.193079 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:41.193044 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b56dd55f-p7gf5" Jul 13 20:20:52.618771 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.618737 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht"] Jul 13 20:20:52.626668 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.626636 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.629301 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.629269 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 13 20:20:52.630303 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.630284 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 13 20:20:52.630501 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.630283 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mqc4w\"" Jul 13 20:20:52.630501 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.630402 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 13 20:20:52.636803 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.636780 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht"] Jul 13 20:20:52.754760 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.754712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-manager-config\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.755131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.755109 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zf42\" (UniqueName: \"kubernetes.io/projected/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-kube-api-access-6zf42\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.755364 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.755345 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-cert\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.755523 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.755502 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-metrics-cert\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.857043 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.857002 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zf42\" (UniqueName: \"kubernetes.io/projected/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-kube-api-access-6zf42\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.857243 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.857067 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-cert\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.857243 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.857090 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-metrics-cert\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.857243 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.857127 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-manager-config\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.857996 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.857969 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-manager-config\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.859961 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.859941 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-metrics-cert\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.860083 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.860040 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-cert\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.873733 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.873671 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zf42\" (UniqueName: \"kubernetes.io/projected/81bce3da-2a6b-48c2-8af3-21d2b0a718b8-kube-api-access-6zf42\") pod \"lws-controller-manager-7f676ff564-sdzht\" (UID: \"81bce3da-2a6b-48c2-8af3-21d2b0a718b8\") " pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:52.939383 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:52.939348 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:53.111781 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:53.111755 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht"] Jul 13 20:20:53.116332 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:20:53.116308 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81bce3da_2a6b_48c2_8af3_21d2b0a718b8.slice/crio-01f9fa58f844652a83d255e1e06534809d7b62c2b6bd1ede1694bd08dce2626c WatchSource:0}: Error finding container 01f9fa58f844652a83d255e1e06534809d7b62c2b6bd1ede1694bd08dce2626c: Status 404 returned error can't find the container with id 01f9fa58f844652a83d255e1e06534809d7b62c2b6bd1ede1694bd08dce2626c Jul 13 20:20:53.290759 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:53.290667 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" event={"ID":"81bce3da-2a6b-48c2-8af3-21d2b0a718b8","Type":"ContainerStarted","Data":"01f9fa58f844652a83d255e1e06534809d7b62c2b6bd1ede1694bd08dce2626c"} Jul 13 20:20:55.302440 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.302388 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" event={"ID":"81bce3da-2a6b-48c2-8af3-21d2b0a718b8","Type":"ContainerStarted","Data":"d4f9115598437d5fea1591c3073715bfa1fb75e0910b238d396b4b6f28acc653"} Jul 13 20:20:55.302878 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.302522 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:20:55.358276 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.358223 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" podStartSLOduration=1.856405715 podStartE2EDuration="3.358197573s" totalImagesPullingTime="1.501791858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:55.351792409 +0000 UTC m=+386.498059570" watchObservedRunningTime="2026-07-13 20:20:55.358197573 +0000 UTC m=+386.504464734" Jul 13 20:20:55.666441 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.666339 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2"] Jul 13 20:20:55.669870 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.669847 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" Jul 13 20:20:55.672307 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.672284 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 13 20:20:55.672470 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.672381 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-f4lrw\"" Jul 13 20:20:55.687541 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.687516 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr"] Jul 13 20:20:55.691605 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.691576 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" Jul 13 20:20:55.693002 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.692981 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2"] Jul 13 20:20:55.697905 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.697881 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wldk5\"" Jul 13 20:20:55.699903 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.699880 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 13 20:20:55.722557 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.722516 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr"] Jul 13 20:20:55.781693 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.781650 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01e36626-7a98-451d-ade1-60059a0e6d7f-cert\") pod \"kserve-controller-manager-d6f4b65b8-pfcq2\" (UID: \"01e36626-7a98-451d-ade1-60059a0e6d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" Jul 13 20:20:55.781693 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.781694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pfhx\" (UniqueName: \"kubernetes.io/projected/0ddbc65d-796f-4dbc-a870-b079c377a278-kube-api-access-5pfhx\") pod \"llmisvc-controller-manager-777c6bd7d5-z6ndr\" (UID: \"0ddbc65d-796f-4dbc-a870-b079c377a278\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" Jul 13 20:20:55.781964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.781802 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ddbc65d-796f-4dbc-a870-b079c377a278-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-z6ndr\" (UID: \"0ddbc65d-796f-4dbc-a870-b079c377a278\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" Jul 13 20:20:55.781964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.781876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm9xv\" (UniqueName: \"kubernetes.io/projected/01e36626-7a98-451d-ade1-60059a0e6d7f-kube-api-access-rm9xv\") pod \"kserve-controller-manager-d6f4b65b8-pfcq2\" (UID: \"01e36626-7a98-451d-ade1-60059a0e6d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" Jul 13 20:20:55.882468 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.882426 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm9xv\" (UniqueName: \"kubernetes.io/projected/01e36626-7a98-451d-ade1-60059a0e6d7f-kube-api-access-rm9xv\") pod \"kserve-controller-manager-d6f4b65b8-pfcq2\" (UID: \"01e36626-7a98-451d-ade1-60059a0e6d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" Jul 13 20:20:55.882651 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.882496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01e36626-7a98-451d-ade1-60059a0e6d7f-cert\") pod \"kserve-controller-manager-d6f4b65b8-pfcq2\" (UID: \"01e36626-7a98-451d-ade1-60059a0e6d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" Jul 13 20:20:55.882651 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.882530 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pfhx\" (UniqueName: \"kubernetes.io/projected/0ddbc65d-796f-4dbc-a870-b079c377a278-kube-api-access-5pfhx\") pod \"llmisvc-controller-manager-777c6bd7d5-z6ndr\" (UID: \"0ddbc65d-796f-4dbc-a870-b079c377a278\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" Jul 13 20:20:55.882651 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.882605 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ddbc65d-796f-4dbc-a870-b079c377a278-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-z6ndr\" (UID: \"0ddbc65d-796f-4dbc-a870-b079c377a278\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" Jul 13 20:20:55.882815 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:20:55.882657 2568 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 13 20:20:55.882815 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:20:55.882713 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e36626-7a98-451d-ade1-60059a0e6d7f-cert podName:01e36626-7a98-451d-ade1-60059a0e6d7f nodeName:}" failed. No retries permitted until 2026-07-13 20:20:56.382698409 +0000 UTC m=+387.528965551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/01e36626-7a98-451d-ade1-60059a0e6d7f-cert") pod "kserve-controller-manager-d6f4b65b8-pfcq2" (UID: "01e36626-7a98-451d-ade1-60059a0e6d7f") : secret "kserve-webhook-server-cert" not found Jul 13 20:20:55.882815 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:20:55.882739 2568 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 13 20:20:55.882815 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:20:55.882799 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ddbc65d-796f-4dbc-a870-b079c377a278-cert podName:0ddbc65d-796f-4dbc-a870-b079c377a278 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:56.382781956 +0000 UTC m=+387.529049108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0ddbc65d-796f-4dbc-a870-b079c377a278-cert") pod "llmisvc-controller-manager-777c6bd7d5-z6ndr" (UID: "0ddbc65d-796f-4dbc-a870-b079c377a278") : secret "llmisvc-webhook-server-cert" not found Jul 13 20:20:55.902087 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.902053 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm9xv\" (UniqueName: \"kubernetes.io/projected/01e36626-7a98-451d-ade1-60059a0e6d7f-kube-api-access-rm9xv\") pod \"kserve-controller-manager-d6f4b65b8-pfcq2\" (UID: \"01e36626-7a98-451d-ade1-60059a0e6d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" Jul 13 20:20:55.906251 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:55.906226 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pfhx\" (UniqueName: \"kubernetes.io/projected/0ddbc65d-796f-4dbc-a870-b079c377a278-kube-api-access-5pfhx\") pod \"llmisvc-controller-manager-777c6bd7d5-z6ndr\" (UID: \"0ddbc65d-796f-4dbc-a870-b079c377a278\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" Jul 13 20:20:56.386747 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:56.386707 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01e36626-7a98-451d-ade1-60059a0e6d7f-cert\") pod \"kserve-controller-manager-d6f4b65b8-pfcq2\" (UID: \"01e36626-7a98-451d-ade1-60059a0e6d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" Jul 13 20:20:56.387279 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:56.386924 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ddbc65d-796f-4dbc-a870-b079c377a278-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-z6ndr\" (UID: \"0ddbc65d-796f-4dbc-a870-b079c377a278\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" Jul 13 20:20:56.389151 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:56.389126 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ddbc65d-796f-4dbc-a870-b079c377a278-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-z6ndr\" (UID: \"0ddbc65d-796f-4dbc-a870-b079c377a278\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" Jul 13 20:20:56.389294 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:56.389273 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01e36626-7a98-451d-ade1-60059a0e6d7f-cert\") pod \"kserve-controller-manager-d6f4b65b8-pfcq2\" (UID: \"01e36626-7a98-451d-ade1-60059a0e6d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" Jul 13 20:20:56.580851 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:56.580814 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" Jul 13 20:20:56.604215 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:56.604185 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" Jul 13 20:20:56.744079 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:56.744051 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2"] Jul 13 20:20:56.745709 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:20:56.745684 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e36626_7a98_451d_ade1_60059a0e6d7f.slice/crio-decd54350735b47bb62ab1de9c8ad980faa6dd260524d170c19365a4fdc76a04 WatchSource:0}: Error finding container decd54350735b47bb62ab1de9c8ad980faa6dd260524d170c19365a4fdc76a04: Status 404 returned error can't find the container with id decd54350735b47bb62ab1de9c8ad980faa6dd260524d170c19365a4fdc76a04 Jul 13 20:20:56.760959 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:56.760938 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr"] Jul 13 20:20:56.762757 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:20:56.762735 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0ddbc65d_796f_4dbc_a870_b079c377a278.slice/crio-db5ff5f00a78a66ff2ddabfb4be64f055fb823633c5ebcbd2a7a3f59230af112 WatchSource:0}: Error finding container db5ff5f00a78a66ff2ddabfb4be64f055fb823633c5ebcbd2a7a3f59230af112: Status 404 returned error can't find the container with id db5ff5f00a78a66ff2ddabfb4be64f055fb823633c5ebcbd2a7a3f59230af112 Jul 13 20:20:57.319778 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:57.319735 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" event={"ID":"01e36626-7a98-451d-ade1-60059a0e6d7f","Type":"ContainerStarted","Data":"decd54350735b47bb62ab1de9c8ad980faa6dd260524d170c19365a4fdc76a04"} Jul 13 20:20:57.320913 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:20:57.320889 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" event={"ID":"0ddbc65d-796f-4dbc-a870-b079c377a278","Type":"ContainerStarted","Data":"db5ff5f00a78a66ff2ddabfb4be64f055fb823633c5ebcbd2a7a3f59230af112"} Jul 13 20:21:00.338446 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:00.338393 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" event={"ID":"01e36626-7a98-451d-ade1-60059a0e6d7f","Type":"ContainerStarted","Data":"f7c32ddde91938d83c218a732968a9e435697af0515fc96f8849275c94b88ce7"} Jul 13 20:21:00.338921 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:00.338539 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" Jul 13 20:21:00.361601 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:00.361548 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" podStartSLOduration=2.561827938 podStartE2EDuration="5.361505892s" totalImagesPullingTime="2.799677954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:21:00.360074275 +0000 UTC m=+391.506341437" watchObservedRunningTime="2026-07-13 20:21:00.361505892 +0000 UTC m=+391.507773054" Jul 13 20:21:01.344566 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:01.344530 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" event={"ID":"0ddbc65d-796f-4dbc-a870-b079c377a278","Type":"ContainerStarted","Data":"4d5fe18905f4b5ee0e65f78fccf7523a7646f8a849c4181215cdd8fda1518afa"} Jul 13 20:21:01.363237 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:01.363189 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" podStartSLOduration=2.183935543 podStartE2EDuration="6.36317445s" totalImagesPullingTime="4.179238907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:21:01.361450496 +0000 UTC m=+392.507717657" watchObservedRunningTime="2026-07-13 20:21:01.36317445 +0000 UTC m=+392.509441804" Jul 13 20:21:02.349257 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:02.349230 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" Jul 13 20:21:06.310456 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:06.310402 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7f676ff564-sdzht" Jul 13 20:21:31.349774 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:31.349745 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pfcq2" Jul 13 20:21:33.356784 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:33.356751 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z6ndr" Jul 13 20:21:44.446121 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.446079 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n5r5m"] Jul 13 20:21:44.453091 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.453065 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" Jul 13 20:21:44.456035 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.456012 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-xqd5r\"" Jul 13 20:21:44.456862 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.456838 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 20:21:44.456993 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.456881 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 20:21:44.465491 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.465466 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n5r5m"] Jul 13 20:21:44.544942 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.544912 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8qk6\" (UniqueName: \"kubernetes.io/projected/76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09-kube-api-access-z8qk6\") pod \"kuadrant-operator-catalog-n5r5m\" (UID: \"76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09\") " pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" Jul 13 20:21:44.646068 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.646037 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8qk6\" (UniqueName: \"kubernetes.io/projected/76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09-kube-api-access-z8qk6\") pod \"kuadrant-operator-catalog-n5r5m\" (UID: \"76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09\") " pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" Jul 13 20:21:44.658938 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.658917 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8qk6\" (UniqueName: \"kubernetes.io/projected/76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09-kube-api-access-z8qk6\") pod \"kuadrant-operator-catalog-n5r5m\" (UID: \"76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09\") " pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" Jul 13 20:21:44.764477 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.764374 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" Jul 13 20:21:44.800137 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.800108 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n5r5m"] Jul 13 20:21:44.920810 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:44.920786 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n5r5m"] Jul 13 20:21:44.923628 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:21:44.923603 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b468ec_6dcb_4a86_bfd9_a9bfb6eeee09.slice/crio-be76a0a69bffc174b8ada5c52661feb41dc730a64ffe3f5ed7a1a548779e7f54 WatchSource:0}: Error finding container be76a0a69bffc174b8ada5c52661feb41dc730a64ffe3f5ed7a1a548779e7f54: Status 404 returned error can't find the container with id be76a0a69bffc174b8ada5c52661feb41dc730a64ffe3f5ed7a1a548779e7f54 Jul 13 20:21:45.011270 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:45.011243 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6rxrv"] Jul 13 20:21:45.014725 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:45.014680 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" Jul 13 20:21:45.023860 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:45.023838 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6rxrv"] Jul 13 20:21:45.150675 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:45.150641 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkjqw\" (UniqueName: \"kubernetes.io/projected/8465cb61-31ce-466a-acdd-6141c94bd720-kube-api-access-gkjqw\") pod \"kuadrant-operator-catalog-6rxrv\" (UID: \"8465cb61-31ce-466a-acdd-6141c94bd720\") " pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" Jul 13 20:21:45.251810 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:45.251783 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkjqw\" (UniqueName: \"kubernetes.io/projected/8465cb61-31ce-466a-acdd-6141c94bd720-kube-api-access-gkjqw\") pod \"kuadrant-operator-catalog-6rxrv\" (UID: \"8465cb61-31ce-466a-acdd-6141c94bd720\") " pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" Jul 13 20:21:45.260513 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:45.260482 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkjqw\" (UniqueName: \"kubernetes.io/projected/8465cb61-31ce-466a-acdd-6141c94bd720-kube-api-access-gkjqw\") pod \"kuadrant-operator-catalog-6rxrv\" (UID: \"8465cb61-31ce-466a-acdd-6141c94bd720\") " pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" Jul 13 20:21:45.324189 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:45.324154 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" Jul 13 20:21:45.458136 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:45.458108 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6rxrv"] Jul 13 20:21:45.459903 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:21:45.459874 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8465cb61_31ce_466a_acdd_6141c94bd720.slice/crio-26814f9a4ecf85aff747e6bc6b085e799b7245a01a337e8731e2c7dbb0dbf85d WatchSource:0}: Error finding container 26814f9a4ecf85aff747e6bc6b085e799b7245a01a337e8731e2c7dbb0dbf85d: Status 404 returned error can't find the container with id 26814f9a4ecf85aff747e6bc6b085e799b7245a01a337e8731e2c7dbb0dbf85d Jul 13 20:21:45.545914 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:45.545878 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" event={"ID":"76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09","Type":"ContainerStarted","Data":"be76a0a69bffc174b8ada5c52661feb41dc730a64ffe3f5ed7a1a548779e7f54"} Jul 13 20:21:45.547249 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:45.547216 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" event={"ID":"8465cb61-31ce-466a-acdd-6141c94bd720","Type":"ContainerStarted","Data":"26814f9a4ecf85aff747e6bc6b085e799b7245a01a337e8731e2c7dbb0dbf85d"} Jul 13 20:21:47.559791 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:47.559754 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" event={"ID":"76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09","Type":"ContainerStarted","Data":"9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b"} Jul 13 20:21:47.560275 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:47.559805 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" podUID="76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09" containerName="registry-server" containerID="cri-o://9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b" gracePeriod=2 Jul 13 20:21:47.561396 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:47.561373 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" event={"ID":"8465cb61-31ce-466a-acdd-6141c94bd720","Type":"ContainerStarted","Data":"c1c821b75c791b3fa5ea76de5533efdfc2ea914409bea340200ab3b6a9f93181"} Jul 13 20:21:47.576674 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:47.576635 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" podStartSLOduration=1.539771156 podStartE2EDuration="3.576619373s" totalImagesPullingTime="2.036848217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:21:47.574625799 +0000 UTC m=+438.720892960" watchObservedRunningTime="2026-07-13 20:21:47.576619373 +0000 UTC m=+438.722886534" Jul 13 20:21:47.588934 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:47.588896 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" podStartSLOduration=2.086129862 podStartE2EDuration="3.588880634s" totalImagesPullingTime="1.502750772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:21:47.58788217 +0000 UTC m=+438.734149334" watchObservedRunningTime="2026-07-13 20:21:47.588880634 +0000 UTC m=+438.735147794" Jul 13 20:21:47.803175 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:47.803155 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" Jul 13 20:21:47.875561 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:47.875488 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8qk6\" (UniqueName: \"kubernetes.io/projected/76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09-kube-api-access-z8qk6\") pod \"76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09\" (UID: \"76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09\") " Jul 13 20:21:47.877663 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:47.877643 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09-kube-api-access-z8qk6" (OuterVolumeSpecName: "kube-api-access-z8qk6") pod "76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09" (UID: "76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09"). InnerVolumeSpecName "kube-api-access-z8qk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:21:47.976920 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:47.976885 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8qk6\" (UniqueName: \"kubernetes.io/projected/76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09-kube-api-access-z8qk6\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:21:48.571541 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:48.571506 2568 generic.go:356] "Generic (PLEG): container finished" podID="76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09" containerID="9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b" exitCode=0 Jul 13 20:21:48.571933 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:48.571556 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" Jul 13 20:21:48.571933 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:48.571585 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" event={"ID":"76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09","Type":"ContainerDied","Data":"9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b"} Jul 13 20:21:48.571933 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:48.571664 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n5r5m" event={"ID":"76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09","Type":"ContainerDied","Data":"be76a0a69bffc174b8ada5c52661feb41dc730a64ffe3f5ed7a1a548779e7f54"} Jul 13 20:21:48.571933 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:48.571690 2568 scope.go:117] "RemoveContainer" containerID="9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b" Jul 13 20:21:48.580864 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:48.580845 2568 scope.go:117] "RemoveContainer" containerID="9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b" Jul 13 20:21:48.581127 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:21:48.581112 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b\": container with ID starting with 9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b not found: ID does not exist" containerID="9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b" Jul 13 20:21:48.581169 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:48.581135 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b"} err="failed to get container status \"9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b\": rpc error: code = NotFound desc = could not find container \"9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b\": container with ID starting with 9675084c60c8f9c0a4476345ec6cf4a638dcb6b6c5e371357cab0856dd9e4d9b not found: ID does not exist" Jul 13 20:21:48.593781 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:48.593758 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n5r5m"] Jul 13 20:21:48.595996 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:48.595973 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n5r5m"] Jul 13 20:21:49.402575 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:49.402541 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09" path="/var/lib/kubelet/pods/76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09/volumes" Jul 13 20:21:55.324480 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:55.324380 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" Jul 13 20:21:55.324480 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:55.324455 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" Jul 13 20:21:55.346703 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:55.346680 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" Jul 13 20:21:55.625507 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:21:55.625427 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6rxrv" Jul 13 20:22:18.290123 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.290083 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b5468d855-77cv6"] Jul 13 20:22:18.290739 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.290709 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09" containerName="registry-server" Jul 13 20:22:18.290739 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.290730 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09" containerName="registry-server" Jul 13 20:22:18.290871 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.290841 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="76b468ec-6dcb-4a86-bfd9-a9bfb6eeee09" containerName="registry-server" Jul 13 20:22:18.295332 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.295302 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.308720 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.308697 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b5468d855-77cv6"] Jul 13 20:22:18.347282 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.347249 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-console-config\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.347511 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.347303 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-trusted-ca-bundle\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.347511 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.347401 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-oauth-serving-cert\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.347511 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.347501 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5cc9cddc-8608-4a81-8669-6538645e91a1-console-serving-cert\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.347686 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.347564 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgqz8\" (UniqueName: \"kubernetes.io/projected/5cc9cddc-8608-4a81-8669-6538645e91a1-kube-api-access-dgqz8\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.347686 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.347610 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5cc9cddc-8608-4a81-8669-6538645e91a1-console-oauth-config\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.347686 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.347636 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-service-ca\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.448363 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.448327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5cc9cddc-8608-4a81-8669-6538645e91a1-console-oauth-config\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.448363 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.448362 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-service-ca\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.448962 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.448577 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-console-config\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.448962 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.448623 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-trusted-ca-bundle\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.448962 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.448648 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-oauth-serving-cert\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.448962 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.448821 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5cc9cddc-8608-4a81-8669-6538645e91a1-console-serving-cert\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.448962 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.448921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgqz8\" (UniqueName: \"kubernetes.io/projected/5cc9cddc-8608-4a81-8669-6538645e91a1-kube-api-access-dgqz8\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.449243 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.449183 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-service-ca\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.449338 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.449313 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-console-config\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.449442 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.449405 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-trusted-ca-bundle\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.449710 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.449691 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5cc9cddc-8608-4a81-8669-6538645e91a1-oauth-serving-cert\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.450881 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.450859 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5cc9cddc-8608-4a81-8669-6538645e91a1-console-oauth-config\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.451489 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.451470 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5cc9cddc-8608-4a81-8669-6538645e91a1-console-serving-cert\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.457031 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.457013 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgqz8\" (UniqueName: \"kubernetes.io/projected/5cc9cddc-8608-4a81-8669-6538645e91a1-kube-api-access-dgqz8\") pod \"console-5b5468d855-77cv6\" (UID: \"5cc9cddc-8608-4a81-8669-6538645e91a1\") " pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.603934 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.603901 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:18.742259 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:18.742235 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b5468d855-77cv6"] Jul 13 20:22:18.744349 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:22:18.744313 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cc9cddc_8608_4a81_8669_6538645e91a1.slice/crio-7fe29c5f7bcdee36d9197fa4c8efb79d1235fa1c6b366900a91a9aeb354acf3c WatchSource:0}: Error finding container 7fe29c5f7bcdee36d9197fa4c8efb79d1235fa1c6b366900a91a9aeb354acf3c: Status 404 returned error can't find the container with id 7fe29c5f7bcdee36d9197fa4c8efb79d1235fa1c6b366900a91a9aeb354acf3c Jul 13 20:22:19.722097 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:19.721285 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b5468d855-77cv6" event={"ID":"5cc9cddc-8608-4a81-8669-6538645e91a1","Type":"ContainerStarted","Data":"2a0a34fc842689e65048b6a0727aaa88ad6a21db8a07d0ef1389226c68b5e31a"} Jul 13 20:22:19.722097 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:19.721344 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b5468d855-77cv6" event={"ID":"5cc9cddc-8608-4a81-8669-6538645e91a1","Type":"ContainerStarted","Data":"7fe29c5f7bcdee36d9197fa4c8efb79d1235fa1c6b366900a91a9aeb354acf3c"} Jul 13 20:22:19.746125 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:19.746070 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b5468d855-77cv6" podStartSLOduration=1.7460533379999998 podStartE2EDuration="1.746053338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:22:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:19.744629148 +0000 UTC m=+470.890896336" watchObservedRunningTime="2026-07-13 20:22:19.746053338 +0000 UTC m=+470.892320498" Jul 13 20:22:21.693080 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:21.693040 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb"] Jul 13 20:22:21.696760 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:21.696737 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:21.699384 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:21.699365 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-42s9q\"" Jul 13 20:22:21.712761 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:21.712741 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb"] Jul 13 20:22:21.783374 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:21.783342 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9mfr\" (UniqueName: \"kubernetes.io/projected/43416db0-59f4-433c-81dc-6615cc503f63-kube-api-access-z9mfr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" (UID: \"43416db0-59f4-433c-81dc-6615cc503f63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:21.783562 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:21.783457 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43416db0-59f4-433c-81dc-6615cc503f63-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" (UID: \"43416db0-59f4-433c-81dc-6615cc503f63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:21.884536 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:21.884502 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9mfr\" (UniqueName: \"kubernetes.io/projected/43416db0-59f4-433c-81dc-6615cc503f63-kube-api-access-z9mfr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" (UID: \"43416db0-59f4-433c-81dc-6615cc503f63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:21.884715 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:21.884591 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43416db0-59f4-433c-81dc-6615cc503f63-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" (UID: \"43416db0-59f4-433c-81dc-6615cc503f63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:21.884998 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:21.884974 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43416db0-59f4-433c-81dc-6615cc503f63-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" (UID: \"43416db0-59f4-433c-81dc-6615cc503f63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:21.895227 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:21.895201 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9mfr\" (UniqueName: \"kubernetes.io/projected/43416db0-59f4-433c-81dc-6615cc503f63-kube-api-access-z9mfr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" (UID: \"43416db0-59f4-433c-81dc-6615cc503f63\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:22.007299 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:22.007215 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:22.144579 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:22.144550 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb"] Jul 13 20:22:22.147322 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:22:22.147288 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43416db0_59f4_433c_81dc_6615cc503f63.slice/crio-a37eb7912739a19cf421e24cbb0d618f4138631b1303b67fe71c1a0a452e75bb WatchSource:0}: Error finding container a37eb7912739a19cf421e24cbb0d618f4138631b1303b67fe71c1a0a452e75bb: Status 404 returned error can't find the container with id a37eb7912739a19cf421e24cbb0d618f4138631b1303b67fe71c1a0a452e75bb Jul 13 20:22:22.744601 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:22.744562 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" event={"ID":"43416db0-59f4-433c-81dc-6615cc503f63","Type":"ContainerStarted","Data":"a37eb7912739a19cf421e24cbb0d618f4138631b1303b67fe71c1a0a452e75bb"} Jul 13 20:22:28.604349 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:28.604301 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:28.604939 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:28.604381 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:28.609918 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:28.609888 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:28.781260 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:28.781232 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b5468d855-77cv6" Jul 13 20:22:28.846047 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:28.846011 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57fb674c9b-qhtzk"] Jul 13 20:22:29.782964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:29.782865 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" event={"ID":"43416db0-59f4-433c-81dc-6615cc503f63","Type":"ContainerStarted","Data":"558cd3d03bb7bd5b07ed9f5627cd682010affb4b0dff791a7ed1f4d1d4e6459b"} Jul 13 20:22:29.806170 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:29.806115 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" podStartSLOduration=1.9563813479999999 podStartE2EDuration="8.806098144s" totalImagesPullingTime="6.849716796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:29.803265844 +0000 UTC m=+480.949533005" watchObservedRunningTime="2026-07-13 20:22:29.806098144 +0000 UTC m=+480.952365306" Jul 13 20:22:30.793741 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:30.793711 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:41.800348 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:41.800312 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:43.467955 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.467918 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt"] Jul 13 20:22:43.472868 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.472848 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:43.486956 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.486933 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt"] Jul 13 20:22:43.495873 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.495850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eac92236-ae02-4bab-a491-fdd3240dcfa1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6d2mt\" (UID: \"eac92236-ae02-4bab-a491-fdd3240dcfa1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:43.496017 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.495881 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qvsr\" (UniqueName: \"kubernetes.io/projected/eac92236-ae02-4bab-a491-fdd3240dcfa1-kube-api-access-9qvsr\") pod \"kuadrant-operator-controller-manager-86878c769f-6d2mt\" (UID: \"eac92236-ae02-4bab-a491-fdd3240dcfa1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:43.596503 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.596469 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eac92236-ae02-4bab-a491-fdd3240dcfa1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6d2mt\" (UID: \"eac92236-ae02-4bab-a491-fdd3240dcfa1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:43.596503 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.596506 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qvsr\" (UniqueName: \"kubernetes.io/projected/eac92236-ae02-4bab-a491-fdd3240dcfa1-kube-api-access-9qvsr\") pod \"kuadrant-operator-controller-manager-86878c769f-6d2mt\" (UID: \"eac92236-ae02-4bab-a491-fdd3240dcfa1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:43.596847 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.596829 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eac92236-ae02-4bab-a491-fdd3240dcfa1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6d2mt\" (UID: \"eac92236-ae02-4bab-a491-fdd3240dcfa1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:43.606682 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.606654 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qvsr\" (UniqueName: \"kubernetes.io/projected/eac92236-ae02-4bab-a491-fdd3240dcfa1-kube-api-access-9qvsr\") pod \"kuadrant-operator-controller-manager-86878c769f-6d2mt\" (UID: \"eac92236-ae02-4bab-a491-fdd3240dcfa1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:43.636381 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.636344 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb"] Jul 13 20:22:43.636600 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.636559 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" podUID="43416db0-59f4-433c-81dc-6615cc503f63" containerName="manager" containerID="cri-o://558cd3d03bb7bd5b07ed9f5627cd682010affb4b0dff791a7ed1f4d1d4e6459b" gracePeriod=2 Jul 13 20:22:43.638567 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:43.638537 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" is forbidden: User \"system:node:ip-10-0-135-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-149.ec2.internal' and this object" podUID="43416db0-59f4-433c-81dc-6615cc503f63" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:43.639454 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.639405 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb"] Jul 13 20:22:43.659720 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.659698 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt"] Jul 13 20:22:43.660077 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.660058 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:43.669971 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.669524 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97"] Jul 13 20:22:43.670153 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.670104 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43416db0-59f4-433c-81dc-6615cc503f63" containerName="manager" Jul 13 20:22:43.670153 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.670121 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="43416db0-59f4-433c-81dc-6615cc503f63" containerName="manager" Jul 13 20:22:43.670282 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.670259 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="43416db0-59f4-433c-81dc-6615cc503f63" containerName="manager" Jul 13 20:22:43.682540 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.682499 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt"] Jul 13 20:22:43.682652 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.682627 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:43.684989 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:43.684957 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" is forbidden: User \"system:node:ip-10-0-135-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-149.ec2.internal' and this object" podUID="43416db0-59f4-433c-81dc-6615cc503f63" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:43.693575 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.693548 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97"] Jul 13 20:22:43.698982 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.698959 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z"] Jul 13 20:22:43.711268 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.711195 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:22:43.734874 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.734849 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z"] Jul 13 20:22:43.778058 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:43.778017 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" is forbidden: User \"system:node:ip-10-0-135-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-149.ec2.internal' and this object" podUID="43416db0-59f4-433c-81dc-6615cc503f63" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:43.798714 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.798681 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsv8c\" (UniqueName: \"kubernetes.io/projected/6072ed02-479d-4fb5-ba92-0ae587c7d35a-kube-api-access-xsv8c\") pod \"kuadrant-operator-controller-manager-86878c769f-mhq6z\" (UID: \"6072ed02-479d-4fb5-ba92-0ae587c7d35a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:22:43.798858 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.798752 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a826d9db-f59f-4eab-8e7f-8787aa9109e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qhd97\" (UID: \"a826d9db-f59f-4eab-8e7f-8787aa9109e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:43.798858 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.798850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6072ed02-479d-4fb5-ba92-0ae587c7d35a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mhq6z\" (UID: \"6072ed02-479d-4fb5-ba92-0ae587c7d35a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:22:43.798935 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.798869 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hddwh\" (UniqueName: \"kubernetes.io/projected/a826d9db-f59f-4eab-8e7f-8787aa9109e0-kube-api-access-hddwh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qhd97\" (UID: \"a826d9db-f59f-4eab-8e7f-8787aa9109e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:43.856804 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.856773 2568 generic.go:356] "Generic (PLEG): container finished" podID="43416db0-59f4-433c-81dc-6615cc503f63" containerID="558cd3d03bb7bd5b07ed9f5627cd682010affb4b0dff791a7ed1f4d1d4e6459b" exitCode=0 Jul 13 20:22:43.889939 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.889912 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:43.892168 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:43.892144 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" is forbidden: User \"system:node:ip-10-0-135-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-149.ec2.internal' and this object" podUID="43416db0-59f4-433c-81dc-6615cc503f63" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:43.899637 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.899604 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6072ed02-479d-4fb5-ba92-0ae587c7d35a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mhq6z\" (UID: \"6072ed02-479d-4fb5-ba92-0ae587c7d35a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:22:43.899754 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.899643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hddwh\" (UniqueName: \"kubernetes.io/projected/a826d9db-f59f-4eab-8e7f-8787aa9109e0-kube-api-access-hddwh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qhd97\" (UID: \"a826d9db-f59f-4eab-8e7f-8787aa9109e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:43.899754 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.899686 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsv8c\" (UniqueName: \"kubernetes.io/projected/6072ed02-479d-4fb5-ba92-0ae587c7d35a-kube-api-access-xsv8c\") pod \"kuadrant-operator-controller-manager-86878c769f-mhq6z\" (UID: \"6072ed02-479d-4fb5-ba92-0ae587c7d35a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:22:43.899863 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.899752 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a826d9db-f59f-4eab-8e7f-8787aa9109e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qhd97\" (UID: \"a826d9db-f59f-4eab-8e7f-8787aa9109e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:43.900035 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.900003 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6072ed02-479d-4fb5-ba92-0ae587c7d35a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mhq6z\" (UID: \"6072ed02-479d-4fb5-ba92-0ae587c7d35a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:22:43.900156 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.900137 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a826d9db-f59f-4eab-8e7f-8787aa9109e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qhd97\" (UID: \"a826d9db-f59f-4eab-8e7f-8787aa9109e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:43.910475 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.910444 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsv8c\" (UniqueName: \"kubernetes.io/projected/6072ed02-479d-4fb5-ba92-0ae587c7d35a-kube-api-access-xsv8c\") pod \"kuadrant-operator-controller-manager-86878c769f-mhq6z\" (UID: \"6072ed02-479d-4fb5-ba92-0ae587c7d35a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:22:43.910575 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:43.910454 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hddwh\" (UniqueName: \"kubernetes.io/projected/a826d9db-f59f-4eab-8e7f-8787aa9109e0-kube-api-access-hddwh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qhd97\" (UID: \"a826d9db-f59f-4eab-8e7f-8787aa9109e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:44.000825 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.000749 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43416db0-59f4-433c-81dc-6615cc503f63-extensions-socket-volume\") pod \"43416db0-59f4-433c-81dc-6615cc503f63\" (UID: \"43416db0-59f4-433c-81dc-6615cc503f63\") " Jul 13 20:22:44.000952 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.000829 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9mfr\" (UniqueName: \"kubernetes.io/projected/43416db0-59f4-433c-81dc-6615cc503f63-kube-api-access-z9mfr\") pod \"43416db0-59f4-433c-81dc-6615cc503f63\" (UID: \"43416db0-59f4-433c-81dc-6615cc503f63\") " Jul 13 20:22:44.001235 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.001195 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43416db0-59f4-433c-81dc-6615cc503f63-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "43416db0-59f4-433c-81dc-6615cc503f63" (UID: "43416db0-59f4-433c-81dc-6615cc503f63"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:44.002850 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.002827 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43416db0-59f4-433c-81dc-6615cc503f63-kube-api-access-z9mfr" (OuterVolumeSpecName: "kube-api-access-z9mfr") pod "43416db0-59f4-433c-81dc-6615cc503f63" (UID: "43416db0-59f4-433c-81dc-6615cc503f63"). InnerVolumeSpecName "kube-api-access-z9mfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:44.018012 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.017984 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:44.025094 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.025074 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:22:44.102368 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.102339 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9mfr\" (UniqueName: \"kubernetes.io/projected/43416db0-59f4-433c-81dc-6615cc503f63-kube-api-access-z9mfr\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:44.102368 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.102367 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43416db0-59f4-433c-81dc-6615cc503f63-extensions-socket-volume\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:44.164082 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.164044 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97"] Jul 13 20:22:44.168398 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:22:44.168374 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda826d9db_f59f_4eab_8e7f_8787aa9109e0.slice/crio-4bbc6566af214e7ecf6b1d0736ecc89c89fae0e26801714b7d9cf64229b8e899 WatchSource:0}: Error finding container 4bbc6566af214e7ecf6b1d0736ecc89c89fae0e26801714b7d9cf64229b8e899: Status 404 returned error can't find the container with id 4bbc6566af214e7ecf6b1d0736ecc89c89fae0e26801714b7d9cf64229b8e899 Jul 13 20:22:44.193092 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.193052 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z"] Jul 13 20:22:44.205551 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:22:44.205521 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6072ed02_479d_4fb5_ba92_0ae587c7d35a.slice/crio-228a50fea7d01265bfb29bf02100b94060541125d911923e2c8dd8a9fd725cd0 WatchSource:0}: Error finding container 228a50fea7d01265bfb29bf02100b94060541125d911923e2c8dd8a9fd725cd0: Status 404 returned error can't find the container with id 228a50fea7d01265bfb29bf02100b94060541125d911923e2c8dd8a9fd725cd0 Jul 13 20:22:44.862705 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.862669 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" event={"ID":"a826d9db-f59f-4eab-8e7f-8787aa9109e0","Type":"ContainerStarted","Data":"5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8"} Jul 13 20:22:44.863157 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.862837 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:44.863157 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.862871 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" event={"ID":"a826d9db-f59f-4eab-8e7f-8787aa9109e0","Type":"ContainerStarted","Data":"4bbc6566af214e7ecf6b1d0736ecc89c89fae0e26801714b7d9cf64229b8e899"} Jul 13 20:22:44.864056 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.864037 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:44.864143 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.864088 2568 scope.go:117] "RemoveContainer" containerID="558cd3d03bb7bd5b07ed9f5627cd682010affb4b0dff791a7ed1f4d1d4e6459b" Jul 13 20:22:44.865546 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:44.865517 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" is forbidden: User \"system:node:ip-10-0-135-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-149.ec2.internal' and this object" podUID="43416db0-59f4-433c-81dc-6615cc503f63" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:44.865703 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.865684 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" event={"ID":"6072ed02-479d-4fb5-ba92-0ae587c7d35a","Type":"ContainerStarted","Data":"5627859f8a9f620026d320d8b33dbbfb4ec8c997190ef08cb585578e3214fa1b"} Jul 13 20:22:44.865791 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.865724 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" event={"ID":"6072ed02-479d-4fb5-ba92-0ae587c7d35a","Type":"ContainerStarted","Data":"228a50fea7d01265bfb29bf02100b94060541125d911923e2c8dd8a9fd725cd0"} Jul 13 20:22:44.865791 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.865784 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:22:44.895960 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:44.895911 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" podStartSLOduration=1.895895918 podStartE2EDuration="1.895895918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:22:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:44.893177051 +0000 UTC m=+496.039444212" watchObservedRunningTime="2026-07-13 20:22:44.895895918 +0000 UTC m=+496.042163078" Jul 13 20:22:44.897086 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:44.897063 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-brvfb\" is forbidden: User \"system:node:ip-10-0-135-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-149.ec2.internal' and this object" podUID="43416db0-59f4-433c-81dc-6615cc503f63" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-brvfb" Jul 13 20:22:45.404731 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:45.404702 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43416db0-59f4-433c-81dc-6615cc503f63" path="/var/lib/kubelet/pods/43416db0-59f4-433c-81dc-6615cc503f63/volumes" Jul 13 20:22:46.206496 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:46.206455 2568 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-6d2mt_kuadrant-system_eac92236-ae02-4bab-a491-fdd3240dcfa1_0(744e4b7aebdc8064cf3565633b5c72cc44b49e8b01a9aedb0ab6eb7e80d17855): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6d2mt to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 13 20:22:46.206911 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:46.206524 2568 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-6d2mt_kuadrant-system_eac92236-ae02-4bab-a491-fdd3240dcfa1_0(744e4b7aebdc8064cf3565633b5c72cc44b49e8b01a9aedb0ab6eb7e80d17855): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6d2mt to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:46.878297 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:46.878267 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:46.886424 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:46.886390 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:46.889270 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:46.889247 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6d2mt\" is forbidden: User \"system:node:ip-10-0-135-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-149.ec2.internal' and this object" podUID="eac92236-ae02-4bab-a491-fdd3240dcfa1" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:46.927270 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:46.927247 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eac92236-ae02-4bab-a491-fdd3240dcfa1-extensions-socket-volume\") pod \"eac92236-ae02-4bab-a491-fdd3240dcfa1\" (UID: \"eac92236-ae02-4bab-a491-fdd3240dcfa1\") " Jul 13 20:22:46.927372 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:46.927359 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qvsr\" (UniqueName: \"kubernetes.io/projected/eac92236-ae02-4bab-a491-fdd3240dcfa1-kube-api-access-9qvsr\") pod \"eac92236-ae02-4bab-a491-fdd3240dcfa1\" (UID: \"eac92236-ae02-4bab-a491-fdd3240dcfa1\") " Jul 13 20:22:46.927518 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:46.927499 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eac92236-ae02-4bab-a491-fdd3240dcfa1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "eac92236-ae02-4bab-a491-fdd3240dcfa1" (UID: "eac92236-ae02-4bab-a491-fdd3240dcfa1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:46.927615 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:46.927603 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eac92236-ae02-4bab-a491-fdd3240dcfa1-extensions-socket-volume\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:46.929288 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:46.929269 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eac92236-ae02-4bab-a491-fdd3240dcfa1-kube-api-access-9qvsr" (OuterVolumeSpecName: "kube-api-access-9qvsr") pod "eac92236-ae02-4bab-a491-fdd3240dcfa1" (UID: "eac92236-ae02-4bab-a491-fdd3240dcfa1"). InnerVolumeSpecName "kube-api-access-9qvsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:47.028555 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:47.028526 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qvsr\" (UniqueName: \"kubernetes.io/projected/eac92236-ae02-4bab-a491-fdd3240dcfa1-kube-api-access-9qvsr\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:47.403398 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:47.403362 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eac92236-ae02-4bab-a491-fdd3240dcfa1" path="/var/lib/kubelet/pods/eac92236-ae02-4bab-a491-fdd3240dcfa1/volumes" Jul 13 20:22:47.883345 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:47.883315 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:47.887498 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:47.887469 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6d2mt\" is forbidden: User \"system:node:ip-10-0-135-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-149.ec2.internal' and this object" podUID="eac92236-ae02-4bab-a491-fdd3240dcfa1" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:49.402105 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:49.402070 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6d2mt\" is forbidden: User \"system:node:ip-10-0-135-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-149.ec2.internal' and this object" podUID="eac92236-ae02-4bab-a491-fdd3240dcfa1" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6d2mt" Jul 13 20:22:53.872076 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:53.872027 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57fb674c9b-qhtzk" podUID="bb7a259a-fe9f-4e87-8c75-6d179bffe340" containerName="console" containerID="cri-o://04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9" gracePeriod=15 Jul 13 20:22:54.121649 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.121627 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:22:54.140179 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.140092 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" podStartSLOduration=11.140074886 podStartE2EDuration="11.140074886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:22:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:44.945184571 +0000 UTC m=+496.091451733" watchObservedRunningTime="2026-07-13 20:22:54.140074886 +0000 UTC m=+505.286342051" Jul 13 20:22:54.193093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.193062 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-oauth-config\") pod \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " Jul 13 20:22:54.193263 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.193116 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-config\") pod \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " Jul 13 20:22:54.193263 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.193136 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lngjx\" (UniqueName: \"kubernetes.io/projected/bb7a259a-fe9f-4e87-8c75-6d179bffe340-kube-api-access-lngjx\") pod \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " Jul 13 20:22:54.193263 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.193199 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-oauth-serving-cert\") pod \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " Jul 13 20:22:54.193263 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.193223 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-serving-cert\") pod \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " Jul 13 20:22:54.193263 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.193258 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-service-ca\") pod \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " Jul 13 20:22:54.193579 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.193287 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-trusted-ca-bundle\") pod \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\" (UID: \"bb7a259a-fe9f-4e87-8c75-6d179bffe340\") " Jul 13 20:22:54.193579 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.193522 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-config" (OuterVolumeSpecName: "console-config") pod "bb7a259a-fe9f-4e87-8c75-6d179bffe340" (UID: "bb7a259a-fe9f-4e87-8c75-6d179bffe340"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:22:54.193718 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.193692 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bb7a259a-fe9f-4e87-8c75-6d179bffe340" (UID: "bb7a259a-fe9f-4e87-8c75-6d179bffe340"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:22:54.193843 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.193798 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-service-ca" (OuterVolumeSpecName: "service-ca") pod "bb7a259a-fe9f-4e87-8c75-6d179bffe340" (UID: "bb7a259a-fe9f-4e87-8c75-6d179bffe340"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:22:54.193843 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.193808 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bb7a259a-fe9f-4e87-8c75-6d179bffe340" (UID: "bb7a259a-fe9f-4e87-8c75-6d179bffe340"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:22:54.195371 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.195349 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb7a259a-fe9f-4e87-8c75-6d179bffe340-kube-api-access-lngjx" (OuterVolumeSpecName: "kube-api-access-lngjx") pod "bb7a259a-fe9f-4e87-8c75-6d179bffe340" (UID: "bb7a259a-fe9f-4e87-8c75-6d179bffe340"). InnerVolumeSpecName "kube-api-access-lngjx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:54.195463 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.195389 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bb7a259a-fe9f-4e87-8c75-6d179bffe340" (UID: "bb7a259a-fe9f-4e87-8c75-6d179bffe340"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:22:54.195505 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.195478 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bb7a259a-fe9f-4e87-8c75-6d179bffe340" (UID: "bb7a259a-fe9f-4e87-8c75-6d179bffe340"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:22:54.294964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.294928 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-oauth-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:54.294964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.294957 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-config\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:54.294964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.294968 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lngjx\" (UniqueName: \"kubernetes.io/projected/bb7a259a-fe9f-4e87-8c75-6d179bffe340-kube-api-access-lngjx\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:54.295192 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.294976 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-oauth-serving-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:54.295192 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.294985 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb7a259a-fe9f-4e87-8c75-6d179bffe340-console-serving-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:54.295192 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.294993 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:54.295192 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.295002 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb7a259a-fe9f-4e87-8c75-6d179bffe340-trusted-ca-bundle\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:54.922648 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.922612 2568 generic.go:356] "Generic (PLEG): container finished" podID="bb7a259a-fe9f-4e87-8c75-6d179bffe340" containerID="04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9" exitCode=0 Jul 13 20:22:54.923081 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.922677 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fb674c9b-qhtzk" Jul 13 20:22:54.923081 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.922680 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fb674c9b-qhtzk" event={"ID":"bb7a259a-fe9f-4e87-8c75-6d179bffe340","Type":"ContainerDied","Data":"04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9"} Jul 13 20:22:54.923081 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.922812 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fb674c9b-qhtzk" event={"ID":"bb7a259a-fe9f-4e87-8c75-6d179bffe340","Type":"ContainerDied","Data":"e4e2d1316ecfd2ceec9ffa91f5a79d90128e6c34b2762b937db79be893869c1c"} Jul 13 20:22:54.923081 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.922831 2568 scope.go:117] "RemoveContainer" containerID="04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9" Jul 13 20:22:54.931638 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.931622 2568 scope.go:117] "RemoveContainer" containerID="04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9" Jul 13 20:22:54.931878 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:54.931857 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9\": container with ID starting with 04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9 not found: ID does not exist" containerID="04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9" Jul 13 20:22:54.931969 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.931885 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9"} err="failed to get container status \"04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9\": rpc error: code = NotFound desc = could not find container \"04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9\": container with ID starting with 04d720ba6fb81afdb1d316205b3d4986b5bc9a8cc125b4335a25359cdd241ca9 not found: ID does not exist" Jul 13 20:22:54.944325 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.944300 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57fb674c9b-qhtzk"] Jul 13 20:22:54.946047 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:54.946029 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57fb674c9b-qhtzk"] Jul 13 20:22:55.403104 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:55.403069 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb7a259a-fe9f-4e87-8c75-6d179bffe340" path="/var/lib/kubelet/pods/bb7a259a-fe9f-4e87-8c75-6d179bffe340/volumes" Jul 13 20:22:55.874553 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:55.874522 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:22:55.874722 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:55.874596 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:55.966055 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:55.966026 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97"] Jul 13 20:22:55.966479 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:55.966268 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" podUID="a826d9db-f59f-4eab-8e7f-8787aa9109e0" containerName="manager" containerID="cri-o://5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8" gracePeriod=10 Jul 13 20:22:56.226273 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.226251 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:56.315785 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.315755 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a826d9db-f59f-4eab-8e7f-8787aa9109e0-extensions-socket-volume\") pod \"a826d9db-f59f-4eab-8e7f-8787aa9109e0\" (UID: \"a826d9db-f59f-4eab-8e7f-8787aa9109e0\") " Jul 13 20:22:56.315987 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.315824 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hddwh\" (UniqueName: \"kubernetes.io/projected/a826d9db-f59f-4eab-8e7f-8787aa9109e0-kube-api-access-hddwh\") pod \"a826d9db-f59f-4eab-8e7f-8787aa9109e0\" (UID: \"a826d9db-f59f-4eab-8e7f-8787aa9109e0\") " Jul 13 20:22:56.316114 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.316094 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a826d9db-f59f-4eab-8e7f-8787aa9109e0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a826d9db-f59f-4eab-8e7f-8787aa9109e0" (UID: "a826d9db-f59f-4eab-8e7f-8787aa9109e0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:22:56.317811 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.317791 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a826d9db-f59f-4eab-8e7f-8787aa9109e0-kube-api-access-hddwh" (OuterVolumeSpecName: "kube-api-access-hddwh") pod "a826d9db-f59f-4eab-8e7f-8787aa9109e0" (UID: "a826d9db-f59f-4eab-8e7f-8787aa9109e0"). InnerVolumeSpecName "kube-api-access-hddwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:56.417139 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.417047 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a826d9db-f59f-4eab-8e7f-8787aa9109e0-extensions-socket-volume\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:56.417139 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.417080 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hddwh\" (UniqueName: \"kubernetes.io/projected/a826d9db-f59f-4eab-8e7f-8787aa9109e0-kube-api-access-hddwh\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:22:56.935330 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.935292 2568 generic.go:356] "Generic (PLEG): container finished" podID="a826d9db-f59f-4eab-8e7f-8787aa9109e0" containerID="5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8" exitCode=0 Jul 13 20:22:56.935330 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.935333 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" Jul 13 20:22:56.935571 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.935380 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" event={"ID":"a826d9db-f59f-4eab-8e7f-8787aa9109e0","Type":"ContainerDied","Data":"5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8"} Jul 13 20:22:56.935571 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.935442 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97" event={"ID":"a826d9db-f59f-4eab-8e7f-8787aa9109e0","Type":"ContainerDied","Data":"4bbc6566af214e7ecf6b1d0736ecc89c89fae0e26801714b7d9cf64229b8e899"} Jul 13 20:22:56.935571 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.935467 2568 scope.go:117] "RemoveContainer" containerID="5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8" Jul 13 20:22:56.944875 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.944858 2568 scope.go:117] "RemoveContainer" containerID="5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8" Jul 13 20:22:56.945088 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:22:56.945072 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8\": container with ID starting with 5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8 not found: ID does not exist" containerID="5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8" Jul 13 20:22:56.945130 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.945096 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8"} err="failed to get container status \"5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8\": rpc error: code = NotFound desc = could not find container \"5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8\": container with ID starting with 5152a0ee8acd4fe6dd7cb9f618074376fa8996adf183990fa2a84ce072a899a8 not found: ID does not exist" Jul 13 20:22:56.962090 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.962068 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97"] Jul 13 20:22:56.970567 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:56.970549 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qhd97"] Jul 13 20:22:57.403228 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:22:57.403199 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a826d9db-f59f-4eab-8e7f-8787aa9109e0" path="/var/lib/kubelet/pods/a826d9db-f59f-4eab-8e7f-8787aa9109e0/volumes" Jul 13 20:23:12.343838 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.343798 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t"] Jul 13 20:23:12.344408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.344387 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a826d9db-f59f-4eab-8e7f-8787aa9109e0" containerName="manager" Jul 13 20:23:12.344408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.344425 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a826d9db-f59f-4eab-8e7f-8787aa9109e0" containerName="manager" Jul 13 20:23:12.344586 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.344443 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb7a259a-fe9f-4e87-8c75-6d179bffe340" containerName="console" Jul 13 20:23:12.344586 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.344453 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb7a259a-fe9f-4e87-8c75-6d179bffe340" containerName="console" Jul 13 20:23:12.344685 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.344587 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb7a259a-fe9f-4e87-8c75-6d179bffe340" containerName="console" Jul 13 20:23:12.344685 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.344602 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a826d9db-f59f-4eab-8e7f-8787aa9109e0" containerName="manager" Jul 13 20:23:12.349146 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.349123 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.352452 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.352404 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 20:23:12.352585 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.352542 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v546t\"" Jul 13 20:23:12.352720 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.352701 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 20:23:12.352987 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.352974 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 20:23:12.362407 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.362387 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t"] Jul 13 20:23:12.473453 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.473395 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.473640 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.473481 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.473640 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.473552 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.473640 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.473588 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.473640 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.473615 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.473640 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.473635 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.473877 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.473660 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.473877 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.473763 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.473877 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.473801 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b2g2\" (UniqueName: \"kubernetes.io/projected/1071df1c-06ea-4b49-9ca9-f1f504a09d75-kube-api-access-2b2g2\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.574616 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.574573 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.574824 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.574653 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.574824 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.574681 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b2g2\" (UniqueName: \"kubernetes.io/projected/1071df1c-06ea-4b49-9ca9-f1f504a09d75-kube-api-access-2b2g2\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.574824 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.574748 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.574824 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.574779 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.575053 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.574830 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.575053 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.574859 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.575053 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.574888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.575053 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.574915 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.575053 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.575033 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.575303 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.575077 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.575303 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.575285 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.575508 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.575360 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.575814 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.575789 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.577099 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.577079 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.577332 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.577311 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.583005 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.582977 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1071df1c-06ea-4b49-9ca9-f1f504a09d75-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.583613 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.583591 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b2g2\" (UniqueName: \"kubernetes.io/projected/1071df1c-06ea-4b49-9ca9-f1f504a09d75-kube-api-access-2b2g2\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8g8t\" (UID: \"1071df1c-06ea-4b49-9ca9-f1f504a09d75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.660073 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.659989 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:12.805115 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:12.801138 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t"] Jul 13 20:23:12.806805 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:23:12.806779 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1071df1c_06ea_4b49_9ca9_f1f504a09d75.slice/crio-1170aaa614416de1fdd71c7a7c85d9eb7ad15893bd27b666f917cff83a43f572 WatchSource:0}: Error finding container 1170aaa614416de1fdd71c7a7c85d9eb7ad15893bd27b666f917cff83a43f572: Status 404 returned error can't find the container with id 1170aaa614416de1fdd71c7a7c85d9eb7ad15893bd27b666f917cff83a43f572 Jul 13 20:23:13.016350 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:13.016255 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" event={"ID":"1071df1c-06ea-4b49-9ca9-f1f504a09d75","Type":"ContainerStarted","Data":"1170aaa614416de1fdd71c7a7c85d9eb7ad15893bd27b666f917cff83a43f572"} Jul 13 20:23:15.594521 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:15.594483 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 20:23:15.594803 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:15.594572 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 20:23:15.594803 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:15.594618 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 20:23:16.033252 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:16.033170 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" event={"ID":"1071df1c-06ea-4b49-9ca9-f1f504a09d75","Type":"ContainerStarted","Data":"208b83f8029594a13a7d4064da0fbdcb605972944b27d269454a15cf19abaaf2"} Jul 13 20:23:16.055401 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:16.055356 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" podStartSLOduration=1.2701019119999999 podStartE2EDuration="4.055341914s" totalImagesPullingTime="2.785240002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:23:16.054298478 +0000 UTC m=+527.200565639" watchObservedRunningTime="2026-07-13 20:23:16.055341914 +0000 UTC m=+527.201609074" Jul 13 20:23:16.660610 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:16.660579 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:16.665277 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:16.665253 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:17.040026 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.039952 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:17.040782 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.040765 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:23:17.122964 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.122933 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wt857"] Jul 13 20:23:17.126661 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.126638 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:17.129226 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.129204 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 20:23:17.129226 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.129215 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lf9fh\"" Jul 13 20:23:17.138619 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.138597 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wt857"] Jul 13 20:23:17.213804 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.213775 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q97x9\" (UniqueName: \"kubernetes.io/projected/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-kube-api-access-q97x9\") pod \"limitador-limitador-64c94fd64f-wt857\" (UID: \"9696d9f7-6765-4a9d-974f-12f5cba7e6d6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:17.213980 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.213837 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-config-file\") pod \"limitador-limitador-64c94fd64f-wt857\" (UID: \"9696d9f7-6765-4a9d-974f-12f5cba7e6d6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:17.219432 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.219395 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wt857"] Jul 13 20:23:17.314811 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.314784 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-config-file\") pod \"limitador-limitador-64c94fd64f-wt857\" (UID: \"9696d9f7-6765-4a9d-974f-12f5cba7e6d6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:17.315003 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.314917 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q97x9\" (UniqueName: \"kubernetes.io/projected/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-kube-api-access-q97x9\") pod \"limitador-limitador-64c94fd64f-wt857\" (UID: \"9696d9f7-6765-4a9d-974f-12f5cba7e6d6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:17.315389 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.315370 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-config-file\") pod \"limitador-limitador-64c94fd64f-wt857\" (UID: \"9696d9f7-6765-4a9d-974f-12f5cba7e6d6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:17.329744 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.329716 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q97x9\" (UniqueName: \"kubernetes.io/projected/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-kube-api-access-q97x9\") pod \"limitador-limitador-64c94fd64f-wt857\" (UID: \"9696d9f7-6765-4a9d-974f-12f5cba7e6d6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:17.438059 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.438024 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:17.594759 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:17.594727 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wt857"] Jul 13 20:23:17.596362 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:23:17.596329 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9696d9f7_6765_4a9d_974f_12f5cba7e6d6.slice/crio-0d8556790288058e7a58b113efd74608d0b84e760760ddd821e0a6de2b8d3737 WatchSource:0}: Error finding container 0d8556790288058e7a58b113efd74608d0b84e760760ddd821e0a6de2b8d3737: Status 404 returned error can't find the container with id 0d8556790288058e7a58b113efd74608d0b84e760760ddd821e0a6de2b8d3737 Jul 13 20:23:18.045835 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:18.045799 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" event={"ID":"9696d9f7-6765-4a9d-974f-12f5cba7e6d6","Type":"ContainerStarted","Data":"0d8556790288058e7a58b113efd74608d0b84e760760ddd821e0a6de2b8d3737"} Jul 13 20:23:21.062909 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:21.062878 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" event={"ID":"9696d9f7-6765-4a9d-974f-12f5cba7e6d6","Type":"ContainerStarted","Data":"cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09"} Jul 13 20:23:21.063293 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:21.062951 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:21.084207 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:21.084160 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" podStartSLOduration=1.166838295 podStartE2EDuration="4.084142346s" totalImagesPullingTime="2.917304051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:23:21.079243235 +0000 UTC m=+532.225510396" watchObservedRunningTime="2026-07-13 20:23:21.084142346 +0000 UTC m=+532.230409507" Jul 13 20:23:32.069898 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:32.069870 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:32.455526 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:32.455437 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wt857"] Jul 13 20:23:32.455676 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:32.455622 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" podUID="9696d9f7-6765-4a9d-974f-12f5cba7e6d6" containerName="limitador" containerID="cri-o://cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09" gracePeriod=30 Jul 13 20:23:33.013139 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.013072 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:33.042096 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.042065 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q97x9\" (UniqueName: \"kubernetes.io/projected/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-kube-api-access-q97x9\") pod \"9696d9f7-6765-4a9d-974f-12f5cba7e6d6\" (UID: \"9696d9f7-6765-4a9d-974f-12f5cba7e6d6\") " Jul 13 20:23:33.042273 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.042180 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-config-file\") pod \"9696d9f7-6765-4a9d-974f-12f5cba7e6d6\" (UID: \"9696d9f7-6765-4a9d-974f-12f5cba7e6d6\") " Jul 13 20:23:33.042523 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.042499 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-config-file" (OuterVolumeSpecName: "config-file") pod "9696d9f7-6765-4a9d-974f-12f5cba7e6d6" (UID: "9696d9f7-6765-4a9d-974f-12f5cba7e6d6"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:23:33.044112 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.044088 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-kube-api-access-q97x9" (OuterVolumeSpecName: "kube-api-access-q97x9") pod "9696d9f7-6765-4a9d-974f-12f5cba7e6d6" (UID: "9696d9f7-6765-4a9d-974f-12f5cba7e6d6"). InnerVolumeSpecName "kube-api-access-q97x9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:23:33.123311 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.123277 2568 generic.go:356] "Generic (PLEG): container finished" podID="9696d9f7-6765-4a9d-974f-12f5cba7e6d6" containerID="cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09" exitCode=0 Jul 13 20:23:33.123769 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.123317 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" Jul 13 20:23:33.123769 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.123334 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" event={"ID":"9696d9f7-6765-4a9d-974f-12f5cba7e6d6","Type":"ContainerDied","Data":"cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09"} Jul 13 20:23:33.123769 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.123386 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wt857" event={"ID":"9696d9f7-6765-4a9d-974f-12f5cba7e6d6","Type":"ContainerDied","Data":"0d8556790288058e7a58b113efd74608d0b84e760760ddd821e0a6de2b8d3737"} Jul 13 20:23:33.123769 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.123444 2568 scope.go:117] "RemoveContainer" containerID="cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09" Jul 13 20:23:33.131874 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.131856 2568 scope.go:117] "RemoveContainer" containerID="cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09" Jul 13 20:23:33.132178 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:23:33.132155 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09\": container with ID starting with cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09 not found: ID does not exist" containerID="cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09" Jul 13 20:23:33.132266 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.132191 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09"} err="failed to get container status \"cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09\": rpc error: code = NotFound desc = could not find container \"cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09\": container with ID starting with cc0977ee7068937c224ac4a2d835bd8f0407d513d44974b7f19d517eb4845c09 not found: ID does not exist" Jul 13 20:23:33.143697 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.143644 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q97x9\" (UniqueName: \"kubernetes.io/projected/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-kube-api-access-q97x9\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:23:33.143697 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.143675 2568 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9696d9f7-6765-4a9d-974f-12f5cba7e6d6-config-file\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:23:33.146085 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.146064 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wt857"] Jul 13 20:23:33.149637 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.149617 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wt857"] Jul 13 20:23:33.408213 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:33.403852 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9696d9f7-6765-4a9d-974f-12f5cba7e6d6" path="/var/lib/kubelet/pods/9696d9f7-6765-4a9d-974f-12f5cba7e6d6/volumes" Jul 13 20:23:38.872263 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.872232 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j629t"] Jul 13 20:23:38.872745 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.872729 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9696d9f7-6765-4a9d-974f-12f5cba7e6d6" containerName="limitador" Jul 13 20:23:38.872794 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.872747 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9696d9f7-6765-4a9d-974f-12f5cba7e6d6" containerName="limitador" Jul 13 20:23:38.872830 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.872814 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9696d9f7-6765-4a9d-974f-12f5cba7e6d6" containerName="limitador" Jul 13 20:23:38.877642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.877622 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" Jul 13 20:23:38.880295 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.880266 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 13 20:23:38.880479 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.880273 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 20:23:38.880650 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.880636 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 20:23:38.881485 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.881456 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-lp6gj\"" Jul 13 20:23:38.885030 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.885003 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j629t"] Jul 13 20:23:38.994572 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.994536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssv5k\" (UniqueName: \"kubernetes.io/projected/b6897745-694f-412d-9faa-c71e0d36dfbf-kube-api-access-ssv5k\") pod \"postgres-69fb4594cf-j629t\" (UID: \"b6897745-694f-412d-9faa-c71e0d36dfbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" Jul 13 20:23:38.994733 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:38.994607 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b6897745-694f-412d-9faa-c71e0d36dfbf-data\") pod \"postgres-69fb4594cf-j629t\" (UID: \"b6897745-694f-412d-9faa-c71e0d36dfbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" Jul 13 20:23:39.096041 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:39.096006 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssv5k\" (UniqueName: \"kubernetes.io/projected/b6897745-694f-412d-9faa-c71e0d36dfbf-kube-api-access-ssv5k\") pod \"postgres-69fb4594cf-j629t\" (UID: \"b6897745-694f-412d-9faa-c71e0d36dfbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" Jul 13 20:23:39.096212 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:39.096093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b6897745-694f-412d-9faa-c71e0d36dfbf-data\") pod \"postgres-69fb4594cf-j629t\" (UID: \"b6897745-694f-412d-9faa-c71e0d36dfbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" Jul 13 20:23:39.096499 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:39.096478 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b6897745-694f-412d-9faa-c71e0d36dfbf-data\") pod \"postgres-69fb4594cf-j629t\" (UID: \"b6897745-694f-412d-9faa-c71e0d36dfbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" Jul 13 20:23:39.105091 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:39.105072 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssv5k\" (UniqueName: \"kubernetes.io/projected/b6897745-694f-412d-9faa-c71e0d36dfbf-kube-api-access-ssv5k\") pod \"postgres-69fb4594cf-j629t\" (UID: \"b6897745-694f-412d-9faa-c71e0d36dfbf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" Jul 13 20:23:39.189625 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:39.189548 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" Jul 13 20:23:39.320980 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:39.320958 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j629t"] Jul 13 20:23:39.322591 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:23:39.322559 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6897745_694f_412d_9faa_c71e0d36dfbf.slice/crio-3beaa9e60ddebe008b3aa2b7954eb4a2f5ea3f8c42631da31d1adabbcaef3034 WatchSource:0}: Error finding container 3beaa9e60ddebe008b3aa2b7954eb4a2f5ea3f8c42631da31d1adabbcaef3034: Status 404 returned error can't find the container with id 3beaa9e60ddebe008b3aa2b7954eb4a2f5ea3f8c42631da31d1adabbcaef3034 Jul 13 20:23:40.162825 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:40.162779 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" event={"ID":"b6897745-694f-412d-9faa-c71e0d36dfbf","Type":"ContainerStarted","Data":"3beaa9e60ddebe008b3aa2b7954eb4a2f5ea3f8c42631da31d1adabbcaef3034"} Jul 13 20:23:44.189180 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:44.189150 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" event={"ID":"b6897745-694f-412d-9faa-c71e0d36dfbf","Type":"ContainerStarted","Data":"4978ce0d3450fc2c4ae44928d668611f26191a94b6b78a705b915c138279c6e2"} Jul 13 20:23:44.189637 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:44.189253 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" Jul 13 20:23:44.204523 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:44.204316 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" podStartSLOduration=1.422258973 podStartE2EDuration="6.204297185s" totalImagesPullingTime="4.782038212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:23:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:23:44.204016223 +0000 UTC m=+555.350283386" watchObservedRunningTime="2026-07-13 20:23:44.204297185 +0000 UTC m=+555.350564347" Jul 13 20:23:50.223374 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:50.223347 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j629t" Jul 13 20:23:57.172578 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.172543 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55445d4447-jdh67"] Jul 13 20:23:57.176085 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.176064 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:23:57.178959 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.178937 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 20:23:57.179274 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.179257 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 20:23:57.179852 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.179840 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pxttn\"" Jul 13 20:23:57.193210 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.193185 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55445d4447-jdh67"] Jul 13 20:23:57.273656 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.273629 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x276w\" (UniqueName: \"kubernetes.io/projected/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-kube-api-access-x276w\") pod \"maas-controller-55445d4447-jdh67\" (UID: \"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2\") " pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:23:57.273656 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.273660 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-webhook-cert\") pod \"maas-controller-55445d4447-jdh67\" (UID: \"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2\") " pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:23:57.374852 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.374820 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x276w\" (UniqueName: \"kubernetes.io/projected/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-kube-api-access-x276w\") pod \"maas-controller-55445d4447-jdh67\" (UID: \"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2\") " pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:23:57.374852 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.374856 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-webhook-cert\") pod \"maas-controller-55445d4447-jdh67\" (UID: \"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2\") " pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:23:57.377286 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.377265 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-webhook-cert\") pod \"maas-controller-55445d4447-jdh67\" (UID: \"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2\") " pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:23:57.385941 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.385919 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x276w\" (UniqueName: \"kubernetes.io/projected/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-kube-api-access-x276w\") pod \"maas-controller-55445d4447-jdh67\" (UID: \"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2\") " pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:23:57.405212 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.405190 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d466f49f4-n6m9b"] Jul 13 20:23:57.408791 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.408771 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:23:57.420297 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.420273 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d466f49f4-n6m9b"] Jul 13 20:23:57.486590 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.486509 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:23:57.576792 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.576765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmswn\" (UniqueName: \"kubernetes.io/projected/c8156e68-91ab-4c3b-a74e-6c8841219efe-kube-api-access-vmswn\") pod \"maas-controller-d466f49f4-n6m9b\" (UID: \"c8156e68-91ab-4c3b-a74e-6c8841219efe\") " pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:23:57.576945 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.576824 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8156e68-91ab-4c3b-a74e-6c8841219efe-webhook-cert\") pod \"maas-controller-d466f49f4-n6m9b\" (UID: \"c8156e68-91ab-4c3b-a74e-6c8841219efe\") " pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:23:57.618331 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.618307 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55445d4447-jdh67"] Jul 13 20:23:57.620564 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:23:57.620536 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f66c15c_da3b_4ab2_ba95_b5ffc2595ff2.slice/crio-735234d757ff530da5a2ed8a2dc3440886bea6ed75e1b8e6efd0c779454ca09d WatchSource:0}: Error finding container 735234d757ff530da5a2ed8a2dc3440886bea6ed75e1b8e6efd0c779454ca09d: Status 404 returned error can't find the container with id 735234d757ff530da5a2ed8a2dc3440886bea6ed75e1b8e6efd0c779454ca09d Jul 13 20:23:57.677974 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.677936 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8156e68-91ab-4c3b-a74e-6c8841219efe-webhook-cert\") pod \"maas-controller-d466f49f4-n6m9b\" (UID: \"c8156e68-91ab-4c3b-a74e-6c8841219efe\") " pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:23:57.678155 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.678054 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmswn\" (UniqueName: \"kubernetes.io/projected/c8156e68-91ab-4c3b-a74e-6c8841219efe-kube-api-access-vmswn\") pod \"maas-controller-d466f49f4-n6m9b\" (UID: \"c8156e68-91ab-4c3b-a74e-6c8841219efe\") " pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:23:57.680211 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.680191 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8156e68-91ab-4c3b-a74e-6c8841219efe-webhook-cert\") pod \"maas-controller-d466f49f4-n6m9b\" (UID: \"c8156e68-91ab-4c3b-a74e-6c8841219efe\") " pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:23:57.686427 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.686395 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmswn\" (UniqueName: \"kubernetes.io/projected/c8156e68-91ab-4c3b-a74e-6c8841219efe-kube-api-access-vmswn\") pod \"maas-controller-d466f49f4-n6m9b\" (UID: \"c8156e68-91ab-4c3b-a74e-6c8841219efe\") " pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:23:57.719888 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.719864 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:23:57.855330 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:57.855308 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d466f49f4-n6m9b"] Jul 13 20:23:57.857091 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:23:57.857066 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8156e68_91ab_4c3b_a74e_6c8841219efe.slice/crio-f3a37ef6d5fd23b0298787f78d6fca88dc1e8cbb2011586187a254fe38c8f881 WatchSource:0}: Error finding container f3a37ef6d5fd23b0298787f78d6fca88dc1e8cbb2011586187a254fe38c8f881: Status 404 returned error can't find the container with id f3a37ef6d5fd23b0298787f78d6fca88dc1e8cbb2011586187a254fe38c8f881 Jul 13 20:23:58.257574 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:58.257539 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55445d4447-jdh67" event={"ID":"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2","Type":"ContainerStarted","Data":"735234d757ff530da5a2ed8a2dc3440886bea6ed75e1b8e6efd0c779454ca09d"} Jul 13 20:23:58.258736 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:23:58.258707 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d466f49f4-n6m9b" event={"ID":"c8156e68-91ab-4c3b-a74e-6c8841219efe","Type":"ContainerStarted","Data":"f3a37ef6d5fd23b0298787f78d6fca88dc1e8cbb2011586187a254fe38c8f881"} Jul 13 20:24:00.272641 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:00.272550 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d466f49f4-n6m9b" event={"ID":"c8156e68-91ab-4c3b-a74e-6c8841219efe","Type":"ContainerStarted","Data":"bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f"} Jul 13 20:24:00.272641 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:00.272619 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:24:00.273946 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:00.273919 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55445d4447-jdh67" event={"ID":"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2","Type":"ContainerStarted","Data":"8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862"} Jul 13 20:24:00.274080 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:00.274015 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:24:00.319288 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:00.319231 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d466f49f4-n6m9b" podStartSLOduration=1.219082862 podStartE2EDuration="3.319214907s" totalImagesPullingTime="2.100132045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:23:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:24:00.316743467 +0000 UTC m=+571.463010627" watchObservedRunningTime="2026-07-13 20:24:00.319214907 +0000 UTC m=+571.465482067" Jul 13 20:24:11.281649 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.281609 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:24:11.282146 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.281914 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:24:11.299372 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.299313 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55445d4447-jdh67" podStartSLOduration=11.973138948999999 podStartE2EDuration="14.299295096s" totalImagesPullingTime="2.326156147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:23:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:24:00.364048305 +0000 UTC m=+571.510315466" watchObservedRunningTime="2026-07-13 20:24:11.299295096 +0000 UTC m=+582.445562258" Jul 13 20:24:11.346340 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.346311 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55445d4447-jdh67"] Jul 13 20:24:11.346629 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.346592 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55445d4447-jdh67" podUID="3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2" containerName="manager" containerID="cri-o://8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862" gracePeriod=30 Jul 13 20:24:11.593269 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.593244 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:24:11.607425 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.607398 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x276w\" (UniqueName: \"kubernetes.io/projected/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-kube-api-access-x276w\") pod \"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2\" (UID: \"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2\") " Jul 13 20:24:11.607566 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.607548 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-webhook-cert\") pod \"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2\" (UID: \"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2\") " Jul 13 20:24:11.609586 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.609561 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2" (UID: "3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:24:11.609668 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.609601 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-kube-api-access-x276w" (OuterVolumeSpecName: "kube-api-access-x276w") pod "3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2" (UID: "3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2"). InnerVolumeSpecName "kube-api-access-x276w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:24:11.709158 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.709107 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x276w\" (UniqueName: \"kubernetes.io/projected/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-kube-api-access-x276w\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:24:11.709158 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:11.709152 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2-webhook-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:24:12.336966 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:12.336935 2568 generic.go:356] "Generic (PLEG): container finished" podID="3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2" containerID="8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862" exitCode=0 Jul 13 20:24:12.337368 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:12.336979 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55445d4447-jdh67" Jul 13 20:24:12.337368 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:12.337009 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55445d4447-jdh67" event={"ID":"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2","Type":"ContainerDied","Data":"8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862"} Jul 13 20:24:12.337368 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:12.337051 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55445d4447-jdh67" event={"ID":"3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2","Type":"ContainerDied","Data":"735234d757ff530da5a2ed8a2dc3440886bea6ed75e1b8e6efd0c779454ca09d"} Jul 13 20:24:12.337368 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:12.337072 2568 scope.go:117] "RemoveContainer" containerID="8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862" Jul 13 20:24:12.346849 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:12.346832 2568 scope.go:117] "RemoveContainer" containerID="8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862" Jul 13 20:24:12.347118 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:24:12.347093 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862\": container with ID starting with 8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862 not found: ID does not exist" containerID="8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862" Jul 13 20:24:12.347210 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:12.347126 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862"} err="failed to get container status \"8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862\": rpc error: code = NotFound desc = could not find container \"8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862\": container with ID starting with 8917cd92c2b6280a3dd73739e25925f5a4ed6d58c8b23020567ec0a85fd11862 not found: ID does not exist" Jul 13 20:24:12.361407 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:12.361375 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55445d4447-jdh67"] Jul 13 20:24:12.365817 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:12.365789 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55445d4447-jdh67"] Jul 13 20:24:13.403130 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:13.403096 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2" path="/var/lib/kubelet/pods/3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2/volumes" Jul 13 20:24:29.301385 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:29.301356 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:24:29.305897 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:29.305875 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:24:32.016366 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.016325 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5889597b4-c6hps"] Jul 13 20:24:32.016797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.016783 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2" containerName="manager" Jul 13 20:24:32.016797 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.016799 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2" containerName="manager" Jul 13 20:24:32.016895 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.016885 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f66c15c-da3b-4ab2-ba95-b5ffc2595ff2" containerName="manager" Jul 13 20:24:32.021590 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.021572 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:32.024769 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.024746 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 13 20:24:32.039240 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.039220 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5889597b4-c6hps"] Jul 13 20:24:32.102017 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.101980 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/916a932f-8302-43c3-a725-c00d0a906d19-maas-api-tls\") pod \"maas-api-5889597b4-c6hps\" (UID: \"916a932f-8302-43c3-a725-c00d0a906d19\") " pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:32.102193 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.102057 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/916a932f-8302-43c3-a725-c00d0a906d19-openshift-service-ca\") pod \"maas-api-5889597b4-c6hps\" (UID: \"916a932f-8302-43c3-a725-c00d0a906d19\") " pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:32.102193 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.102115 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr96m\" (UniqueName: \"kubernetes.io/projected/916a932f-8302-43c3-a725-c00d0a906d19-kube-api-access-sr96m\") pod \"maas-api-5889597b4-c6hps\" (UID: \"916a932f-8302-43c3-a725-c00d0a906d19\") " pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:32.203157 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.203122 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/916a932f-8302-43c3-a725-c00d0a906d19-maas-api-tls\") pod \"maas-api-5889597b4-c6hps\" (UID: \"916a932f-8302-43c3-a725-c00d0a906d19\") " pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:32.203334 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.203181 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/916a932f-8302-43c3-a725-c00d0a906d19-openshift-service-ca\") pod \"maas-api-5889597b4-c6hps\" (UID: \"916a932f-8302-43c3-a725-c00d0a906d19\") " pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:32.203334 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.203219 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr96m\" (UniqueName: \"kubernetes.io/projected/916a932f-8302-43c3-a725-c00d0a906d19-kube-api-access-sr96m\") pod \"maas-api-5889597b4-c6hps\" (UID: \"916a932f-8302-43c3-a725-c00d0a906d19\") " pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:32.203947 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.203928 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/916a932f-8302-43c3-a725-c00d0a906d19-openshift-service-ca\") pod \"maas-api-5889597b4-c6hps\" (UID: \"916a932f-8302-43c3-a725-c00d0a906d19\") " pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:32.205613 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.205596 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/916a932f-8302-43c3-a725-c00d0a906d19-maas-api-tls\") pod \"maas-api-5889597b4-c6hps\" (UID: \"916a932f-8302-43c3-a725-c00d0a906d19\") " pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:32.222363 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.222334 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr96m\" (UniqueName: \"kubernetes.io/projected/916a932f-8302-43c3-a725-c00d0a906d19-kube-api-access-sr96m\") pod \"maas-api-5889597b4-c6hps\" (UID: \"916a932f-8302-43c3-a725-c00d0a906d19\") " pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:32.331593 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.331557 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:32.477926 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.477902 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5889597b4-c6hps"] Jul 13 20:24:32.479982 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:24:32.479952 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod916a932f_8302_43c3_a725_c00d0a906d19.slice/crio-2375d372fe08754ab6ec0050904f15dd264cb32814454dc7f27e555eb7aa6857 WatchSource:0}: Error finding container 2375d372fe08754ab6ec0050904f15dd264cb32814454dc7f27e555eb7aa6857: Status 404 returned error can't find the container with id 2375d372fe08754ab6ec0050904f15dd264cb32814454dc7f27e555eb7aa6857 Jul 13 20:24:32.708524 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.708446 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-2zm5j"] Jul 13 20:24:32.713511 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.713488 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" Jul 13 20:24:32.715895 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.715875 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6k242\"" Jul 13 20:24:32.716254 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.716237 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 13 20:24:32.721839 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.721818 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-2zm5j"] Jul 13 20:24:32.764452 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.764382 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm"] Jul 13 20:24:32.769120 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.769100 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" Jul 13 20:24:32.782106 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.782081 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm"] Jul 13 20:24:32.808449 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.808398 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lvnc\" (UniqueName: \"kubernetes.io/projected/f508a9e7-526d-449a-9298-562e1f74b2fa-kube-api-access-9lvnc\") pod \"payload-processing-f9b8d9b9b-2zm5j\" (UID: \"f508a9e7-526d-449a-9298-562e1f74b2fa\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" Jul 13 20:24:32.808607 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.808559 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f508a9e7-526d-449a-9298-562e1f74b2fa-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-2zm5j\" (UID: \"f508a9e7-526d-449a-9298-562e1f74b2fa\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" Jul 13 20:24:32.909097 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.909061 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lvnc\" (UniqueName: \"kubernetes.io/projected/f508a9e7-526d-449a-9298-562e1f74b2fa-kube-api-access-9lvnc\") pod \"payload-processing-f9b8d9b9b-2zm5j\" (UID: \"f508a9e7-526d-449a-9298-562e1f74b2fa\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" Jul 13 20:24:32.909261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.909138 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f508a9e7-526d-449a-9298-562e1f74b2fa-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-2zm5j\" (UID: \"f508a9e7-526d-449a-9298-562e1f74b2fa\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" Jul 13 20:24:32.909261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.909165 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/301e4b2f-38c4-44df-be14-3ffa9238e143-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-nt7xm\" (UID: \"301e4b2f-38c4-44df-be14-3ffa9238e143\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" Jul 13 20:24:32.909261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.909210 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fwfp\" (UniqueName: \"kubernetes.io/projected/301e4b2f-38c4-44df-be14-3ffa9238e143-kube-api-access-2fwfp\") pod \"payload-pre-processing-74584bb9f8-nt7xm\" (UID: \"301e4b2f-38c4-44df-be14-3ffa9238e143\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" Jul 13 20:24:32.909852 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.909831 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f508a9e7-526d-449a-9298-562e1f74b2fa-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-2zm5j\" (UID: \"f508a9e7-526d-449a-9298-562e1f74b2fa\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" Jul 13 20:24:32.917718 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:32.917696 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lvnc\" (UniqueName: \"kubernetes.io/projected/f508a9e7-526d-449a-9298-562e1f74b2fa-kube-api-access-9lvnc\") pod \"payload-processing-f9b8d9b9b-2zm5j\" (UID: \"f508a9e7-526d-449a-9298-562e1f74b2fa\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" Jul 13 20:24:33.009839 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:33.009746 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/301e4b2f-38c4-44df-be14-3ffa9238e143-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-nt7xm\" (UID: \"301e4b2f-38c4-44df-be14-3ffa9238e143\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" Jul 13 20:24:33.009839 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:33.009804 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fwfp\" (UniqueName: \"kubernetes.io/projected/301e4b2f-38c4-44df-be14-3ffa9238e143-kube-api-access-2fwfp\") pod \"payload-pre-processing-74584bb9f8-nt7xm\" (UID: \"301e4b2f-38c4-44df-be14-3ffa9238e143\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" Jul 13 20:24:33.010509 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:33.010491 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/301e4b2f-38c4-44df-be14-3ffa9238e143-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-nt7xm\" (UID: \"301e4b2f-38c4-44df-be14-3ffa9238e143\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" Jul 13 20:24:33.017661 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:33.017637 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fwfp\" (UniqueName: \"kubernetes.io/projected/301e4b2f-38c4-44df-be14-3ffa9238e143-kube-api-access-2fwfp\") pod \"payload-pre-processing-74584bb9f8-nt7xm\" (UID: \"301e4b2f-38c4-44df-be14-3ffa9238e143\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" Jul 13 20:24:33.025522 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:33.025500 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" Jul 13 20:24:33.079256 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:33.079171 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" Jul 13 20:24:33.177080 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:33.176896 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-2zm5j"] Jul 13 20:24:33.178937 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:24:33.178748 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf508a9e7_526d_449a_9298_562e1f74b2fa.slice/crio-b97ac70804ba648c5f9292a6f656c02dbe2664406fb98970127f802acf73396f WatchSource:0}: Error finding container b97ac70804ba648c5f9292a6f656c02dbe2664406fb98970127f802acf73396f: Status 404 returned error can't find the container with id b97ac70804ba648c5f9292a6f656c02dbe2664406fb98970127f802acf73396f Jul 13 20:24:33.239410 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:33.239379 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm"] Jul 13 20:24:33.242247 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:24:33.242208 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod301e4b2f_38c4_44df_be14_3ffa9238e143.slice/crio-47b803a197a678b69198cb4fdeaf77e4a019028df75df3fcbf88efbf9bdb5db1 WatchSource:0}: Error finding container 47b803a197a678b69198cb4fdeaf77e4a019028df75df3fcbf88efbf9bdb5db1: Status 404 returned error can't find the container with id 47b803a197a678b69198cb4fdeaf77e4a019028df75df3fcbf88efbf9bdb5db1 Jul 13 20:24:33.464665 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:33.464625 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" event={"ID":"916a932f-8302-43c3-a725-c00d0a906d19","Type":"ContainerStarted","Data":"2375d372fe08754ab6ec0050904f15dd264cb32814454dc7f27e555eb7aa6857"} Jul 13 20:24:33.466737 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:33.466681 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" event={"ID":"301e4b2f-38c4-44df-be14-3ffa9238e143","Type":"ContainerStarted","Data":"47b803a197a678b69198cb4fdeaf77e4a019028df75df3fcbf88efbf9bdb5db1"} Jul 13 20:24:33.468565 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:33.468537 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" event={"ID":"f508a9e7-526d-449a-9298-562e1f74b2fa","Type":"ContainerStarted","Data":"b97ac70804ba648c5f9292a6f656c02dbe2664406fb98970127f802acf73396f"} Jul 13 20:24:35.481481 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:35.481443 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" event={"ID":"301e4b2f-38c4-44df-be14-3ffa9238e143","Type":"ContainerStarted","Data":"e00c24003a0e61a669b563d4ff177c8730b8092dbe08a743c064b8fc176b7c98"} Jul 13 20:24:35.481996 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:35.481633 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" Jul 13 20:24:35.483019 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:35.482994 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" event={"ID":"f508a9e7-526d-449a-9298-562e1f74b2fa","Type":"ContainerStarted","Data":"3a85d55028427e8aa6a53508b9e60477b59dd37e947479a26ab33c7b251b2efb"} Jul 13 20:24:35.483141 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:35.483044 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" Jul 13 20:24:35.484497 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:35.484466 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" event={"ID":"916a932f-8302-43c3-a725-c00d0a906d19","Type":"ContainerStarted","Data":"6fac69282d2ce89807d5316a6709f173a44e58b2ec6553ddc3908bc51a8f76a0"} Jul 13 20:24:35.484607 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:35.484549 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:35.499888 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:35.499849 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" podStartSLOduration=1.756765884 podStartE2EDuration="3.49983533s" totalImagesPullingTime="1.743069446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:24:35.497675005 +0000 UTC m=+606.643942167" watchObservedRunningTime="2026-07-13 20:24:35.49983533 +0000 UTC m=+606.646102491" Jul 13 20:24:35.517267 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:35.517215 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" podStartSLOduration=1.711664531 podStartE2EDuration="3.517200363s" totalImagesPullingTime="1.805535832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:24:35.515113245 +0000 UTC m=+606.661380406" watchObservedRunningTime="2026-07-13 20:24:35.517200363 +0000 UTC m=+606.663467525" Jul 13 20:24:35.541485 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:35.541401 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" podStartSLOduration=2.12451886 podStartE2EDuration="4.541380169s" totalImagesPullingTime="2.416861309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:24:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:24:35.538553116 +0000 UTC m=+606.684820290" watchObservedRunningTime="2026-07-13 20:24:35.541380169 +0000 UTC m=+606.687647335" Jul 13 20:24:40.513014 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:40.512930 2568 generic.go:356] "Generic (PLEG): container finished" podID="1071df1c-06ea-4b49-9ca9-f1f504a09d75" containerID="208b83f8029594a13a7d4064da0fbdcb605972944b27d269454a15cf19abaaf2" exitCode=0 Jul 13 20:24:40.513014 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:40.513003 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" event={"ID":"1071df1c-06ea-4b49-9ca9-f1f504a09d75","Type":"ContainerDied","Data":"208b83f8029594a13a7d4064da0fbdcb605972944b27d269454a15cf19abaaf2"} Jul 13 20:24:40.513502 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:40.513388 2568 scope.go:117] "RemoveContainer" containerID="208b83f8029594a13a7d4064da0fbdcb605972944b27d269454a15cf19abaaf2" Jul 13 20:24:40.514363 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:40.514331 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 20:24:40.514509 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:40.514431 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 20:24:40.514509 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:40.514481 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 20:24:41.495214 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:41.495189 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5889597b4-c6hps" Jul 13 20:24:41.519524 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:41.519497 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" event={"ID":"1071df1c-06ea-4b49-9ca9-f1f504a09d75","Type":"ContainerStarted","Data":"d70633f6eeb8809aeee1e3e30c7314570cd6c0413933c6d46fbf64bfa55d2e45"} Jul 13 20:24:41.661155 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:41.661118 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:24:41.661368 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:41.661342 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-z8g8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 13 20:24:41.661466 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:41.661429 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" podUID="1071df1c-06ea-4b49-9ca9-f1f504a09d75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 13 20:24:42.660908 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:42.660868 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-z8g8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 13 20:24:42.661337 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:42.660942 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" podUID="1071df1c-06ea-4b49-9ca9-f1f504a09d75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 13 20:24:43.660941 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:43.660905 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-z8g8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 13 20:24:43.661352 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:43.660962 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" podUID="1071df1c-06ea-4b49-9ca9-f1f504a09d75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 13 20:24:44.660741 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:44.660695 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-z8g8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 13 20:24:44.660950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:44.660765 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" podUID="1071df1c-06ea-4b49-9ca9-f1f504a09d75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 13 20:24:45.664794 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:45.664767 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:24:45.665242 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:45.665050 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:24:45.665753 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:45.665734 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8g8t" Jul 13 20:24:46.492275 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:46.492247 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-2zm5j" Jul 13 20:24:46.492609 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:24:46.492589 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-nt7xm" Jul 13 20:25:12.785265 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.785231 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk"] Jul 13 20:25:12.790510 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.790487 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:12.793018 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.792978 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 20:25:12.793910 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.793890 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 20:25:12.794013 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.793922 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 13 20:25:12.794013 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.793967 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wnn2k\"" Jul 13 20:25:12.799841 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.799814 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk"] Jul 13 20:25:12.901831 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.901800 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:12.901831 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.901834 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:12.902089 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.901864 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:12.902089 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.901949 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h895v\" (UniqueName: \"kubernetes.io/projected/3db964e8-9153-4bbb-974d-fbea7ecacb04-kube-api-access-h895v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:12.902089 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.901974 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3db964e8-9153-4bbb-974d-fbea7ecacb04-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:12.902089 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:12.902015 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.003162 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.003128 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.003356 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.003167 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.003356 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.003212 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.003356 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.003259 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h895v\" (UniqueName: \"kubernetes.io/projected/3db964e8-9153-4bbb-974d-fbea7ecacb04-kube-api-access-h895v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.003356 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.003284 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3db964e8-9153-4bbb-974d-fbea7ecacb04-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.003653 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.003390 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.003653 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.003587 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.003653 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.003602 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.003817 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.003800 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.005524 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.005503 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3db964e8-9153-4bbb-974d-fbea7ecacb04-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.005914 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.005895 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3db964e8-9153-4bbb-974d-fbea7ecacb04-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.011821 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.011799 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h895v\" (UniqueName: \"kubernetes.io/projected/3db964e8-9153-4bbb-974d-fbea7ecacb04-kube-api-access-h895v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk\" (UID: \"3db964e8-9153-4bbb-974d-fbea7ecacb04\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.102080 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.102052 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:13.245230 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.245197 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk"] Jul 13 20:25:13.246773 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:25:13.246745 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db964e8_9153_4bbb_974d_fbea7ecacb04.slice/crio-8bd51aaae5337486f294a557c6ae116072c4a85ff7809f37fe31a890450673e7 WatchSource:0}: Error finding container 8bd51aaae5337486f294a557c6ae116072c4a85ff7809f37fe31a890450673e7: Status 404 returned error can't find the container with id 8bd51aaae5337486f294a557c6ae116072c4a85ff7809f37fe31a890450673e7 Jul 13 20:25:13.248132 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.248108 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:25:13.693187 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:13.693140 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" event={"ID":"3db964e8-9153-4bbb-974d-fbea7ecacb04","Type":"ContainerStarted","Data":"8bd51aaae5337486f294a557c6ae116072c4a85ff7809f37fe31a890450673e7"} Jul 13 20:25:14.699294 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:14.699260 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" event={"ID":"3db964e8-9153-4bbb-974d-fbea7ecacb04","Type":"ContainerStarted","Data":"98120c59cd4ecfbeefcec0988d9858e99340fcb46d65b61e4fcc9d47c9111f21"} Jul 13 20:25:14.718508 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:14.718394 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" podStartSLOduration=1.49910031 podStartE2EDuration="2.718379147s" totalImagesPullingTime="1.219278837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:25:14.715830802 +0000 UTC m=+645.862097963" watchObservedRunningTime="2026-07-13 20:25:14.718379147 +0000 UTC m=+645.864646308" Jul 13 20:25:23.102243 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:23.102200 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:23.102724 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:23.102275 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:23.115081 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:23.115058 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:23.755610 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:23.755585 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk" Jul 13 20:25:26.879739 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:26.879712 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd"] Jul 13 20:25:26.883734 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:26.883712 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:26.886623 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:26.886601 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 13 20:25:26.898506 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:26.898481 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd"] Jul 13 20:25:26.947458 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:26.947408 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6twxz\" (UniqueName: \"kubernetes.io/projected/08a87a37-0b60-4e14-b135-1c467565e46a-kube-api-access-6twxz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:26.947642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:26.947476 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08a87a37-0b60-4e14-b135-1c467565e46a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:26.947642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:26.947504 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:26.947642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:26.947525 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:26.947642 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:26.947624 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:26.947826 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:26.947668 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.048965 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.048930 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.049157 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.048996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.049157 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.049113 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6twxz\" (UniqueName: \"kubernetes.io/projected/08a87a37-0b60-4e14-b135-1c467565e46a-kube-api-access-6twxz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.049157 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.049149 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08a87a37-0b60-4e14-b135-1c467565e46a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.049365 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.049175 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.049365 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.049199 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.049607 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.049575 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.049743 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.049587 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.049743 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.049704 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.051274 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.051255 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08a87a37-0b60-4e14-b135-1c467565e46a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.051563 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.051547 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08a87a37-0b60-4e14-b135-1c467565e46a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.057495 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.057475 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6twxz\" (UniqueName: \"kubernetes.io/projected/08a87a37-0b60-4e14-b135-1c467565e46a-kube-api-access-6twxz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ckqxd\" (UID: \"08a87a37-0b60-4e14-b135-1c467565e46a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.195691 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.195588 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:27.336746 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.336705 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd"] Jul 13 20:25:27.337889 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:25:27.337854 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a87a37_0b60_4e14_b135_1c467565e46a.slice/crio-ecd3730bb34b538eceefd8b388512c2d2d180ed816de754757e97bdb970dcc84 WatchSource:0}: Error finding container ecd3730bb34b538eceefd8b388512c2d2d180ed816de754757e97bdb970dcc84: Status 404 returned error can't find the container with id ecd3730bb34b538eceefd8b388512c2d2d180ed816de754757e97bdb970dcc84 Jul 13 20:25:27.765938 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.765897 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" event={"ID":"08a87a37-0b60-4e14-b135-1c467565e46a","Type":"ContainerStarted","Data":"592fa138233feac515cd98a7b0f1620db23e9c3fb0d4f22dc71e3e8847361f56"} Jul 13 20:25:27.766107 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.765962 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" event={"ID":"08a87a37-0b60-4e14-b135-1c467565e46a","Type":"ContainerStarted","Data":"ecd3730bb34b538eceefd8b388512c2d2d180ed816de754757e97bdb970dcc84"} Jul 13 20:25:27.786007 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:27.785907 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" podStartSLOduration=1.5957307040000002 podStartE2EDuration="1.785892491s" totalImagesPullingTime="190.161787ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:25:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:25:27.784241671 +0000 UTC m=+658.930508835" watchObservedRunningTime="2026-07-13 20:25:27.785892491 +0000 UTC m=+658.932159651" Jul 13 20:25:37.196060 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:37.196024 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:37.196444 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:37.196092 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:37.208467 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:37.208444 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:37.829011 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:37.828981 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ckqxd" Jul 13 20:25:42.699769 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.699735 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz"] Jul 13 20:25:42.705264 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.705242 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.707800 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.707777 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 13 20:25:42.714777 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.714751 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz"] Jul 13 20:25:42.813376 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.813342 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nndlg\" (UniqueName: \"kubernetes.io/projected/346edc07-31a5-417c-9135-c9e8b559c559-kube-api-access-nndlg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.813582 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.813396 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.813582 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.813435 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/346edc07-31a5-417c-9135-c9e8b559c559-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.813582 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.813514 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.813709 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.813615 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.813709 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.813645 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.914335 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.914291 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.914335 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.914333 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.914588 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.914373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nndlg\" (UniqueName: \"kubernetes.io/projected/346edc07-31a5-417c-9135-c9e8b559c559-kube-api-access-nndlg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.914588 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.914437 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.914588 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.914460 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/346edc07-31a5-417c-9135-c9e8b559c559-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.914588 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.914496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.914808 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.914787 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.914855 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.914829 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.914898 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.914861 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.916926 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.916894 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/346edc07-31a5-417c-9135-c9e8b559c559-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.917139 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.917117 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/346edc07-31a5-417c-9135-c9e8b559c559-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:42.927248 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:42.927219 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nndlg\" (UniqueName: \"kubernetes.io/projected/346edc07-31a5-417c-9135-c9e8b559c559-kube-api-access-nndlg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz\" (UID: \"346edc07-31a5-417c-9135-c9e8b559c559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:43.017671 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:43.017578 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:43.161093 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:43.161036 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz"] Jul 13 20:25:43.163573 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:25:43.163538 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346edc07_31a5_417c_9135_c9e8b559c559.slice/crio-96e09db075ea28f636f61a412931437aaa725aba0e8f362d770b98605ffd8341 WatchSource:0}: Error finding container 96e09db075ea28f636f61a412931437aaa725aba0e8f362d770b98605ffd8341: Status 404 returned error can't find the container with id 96e09db075ea28f636f61a412931437aaa725aba0e8f362d770b98605ffd8341 Jul 13 20:25:43.852469 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:43.852407 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" event={"ID":"346edc07-31a5-417c-9135-c9e8b559c559","Type":"ContainerStarted","Data":"922b977e70652ec9d824bdacf32b44d8f4cf808dd817de3a8302a01382f9ea23"} Jul 13 20:25:43.852872 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:43.852482 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" event={"ID":"346edc07-31a5-417c-9135-c9e8b559c559","Type":"ContainerStarted","Data":"96e09db075ea28f636f61a412931437aaa725aba0e8f362d770b98605ffd8341"} Jul 13 20:25:43.871264 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:43.871214 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" podStartSLOduration=1.656985185 podStartE2EDuration="1.871196591s" totalImagesPullingTime="214.211406ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:25:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:25:43.869505681 +0000 UTC m=+675.015772842" watchObservedRunningTime="2026-07-13 20:25:43.871196591 +0000 UTC m=+675.017463752" Jul 13 20:25:53.017935 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:53.017901 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:53.018346 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:53.017968 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:53.030654 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:53.030622 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:25:53.914547 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:25:53.914517 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz" Jul 13 20:28:10.207122 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.207044 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d466f49f4-n6m9b"] Jul 13 20:28:10.207626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.207246 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-d466f49f4-n6m9b" podUID="c8156e68-91ab-4c3b-a74e-6c8841219efe" containerName="manager" containerID="cri-o://bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f" gracePeriod=30 Jul 13 20:28:10.467793 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.467732 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:28:10.610001 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.609976 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmswn\" (UniqueName: \"kubernetes.io/projected/c8156e68-91ab-4c3b-a74e-6c8841219efe-kube-api-access-vmswn\") pod \"c8156e68-91ab-4c3b-a74e-6c8841219efe\" (UID: \"c8156e68-91ab-4c3b-a74e-6c8841219efe\") " Jul 13 20:28:10.610167 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.610043 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8156e68-91ab-4c3b-a74e-6c8841219efe-webhook-cert\") pod \"c8156e68-91ab-4c3b-a74e-6c8841219efe\" (UID: \"c8156e68-91ab-4c3b-a74e-6c8841219efe\") " Jul 13 20:28:10.611887 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.611856 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8156e68-91ab-4c3b-a74e-6c8841219efe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c8156e68-91ab-4c3b-a74e-6c8841219efe" (UID: "c8156e68-91ab-4c3b-a74e-6c8841219efe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:28:10.612008 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.611976 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8156e68-91ab-4c3b-a74e-6c8841219efe-kube-api-access-vmswn" (OuterVolumeSpecName: "kube-api-access-vmswn") pod "c8156e68-91ab-4c3b-a74e-6c8841219efe" (UID: "c8156e68-91ab-4c3b-a74e-6c8841219efe"). InnerVolumeSpecName "kube-api-access-vmswn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:28:10.627481 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.627456 2568 generic.go:356] "Generic (PLEG): container finished" podID="c8156e68-91ab-4c3b-a74e-6c8841219efe" containerID="bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f" exitCode=0 Jul 13 20:28:10.627609 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.627508 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d466f49f4-n6m9b" Jul 13 20:28:10.627609 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.627510 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d466f49f4-n6m9b" event={"ID":"c8156e68-91ab-4c3b-a74e-6c8841219efe","Type":"ContainerDied","Data":"bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f"} Jul 13 20:28:10.627609 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.627587 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d466f49f4-n6m9b" event={"ID":"c8156e68-91ab-4c3b-a74e-6c8841219efe","Type":"ContainerDied","Data":"f3a37ef6d5fd23b0298787f78d6fca88dc1e8cbb2011586187a254fe38c8f881"} Jul 13 20:28:10.627774 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.627620 2568 scope.go:117] "RemoveContainer" containerID="bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f" Jul 13 20:28:10.636276 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.636261 2568 scope.go:117] "RemoveContainer" containerID="bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f" Jul 13 20:28:10.636582 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:28:10.636560 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f\": container with ID starting with bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f not found: ID does not exist" containerID="bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f" Jul 13 20:28:10.636663 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.636593 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f"} err="failed to get container status \"bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f\": rpc error: code = NotFound desc = could not find container \"bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f\": container with ID starting with bf0a3b617f9d90f0ec3aca7338ba2a353026412828a4ea2fbdb16f8c9a941d8f not found: ID does not exist" Jul 13 20:28:10.650467 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.650445 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d466f49f4-n6m9b"] Jul 13 20:28:10.655182 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.655160 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d466f49f4-n6m9b"] Jul 13 20:28:10.710802 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.710772 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmswn\" (UniqueName: \"kubernetes.io/projected/c8156e68-91ab-4c3b-a74e-6c8841219efe-kube-api-access-vmswn\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:28:10.710802 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:10.710799 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8156e68-91ab-4c3b-a74e-6c8841219efe-webhook-cert\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:28:11.404466 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.404408 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8156e68-91ab-4c3b-a74e-6c8841219efe" path="/var/lib/kubelet/pods/c8156e68-91ab-4c3b-a74e-6c8841219efe/volumes" Jul 13 20:28:11.440551 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.440520 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d466f49f4-wgb26"] Jul 13 20:28:11.440953 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.440940 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8156e68-91ab-4c3b-a74e-6c8841219efe" containerName="manager" Jul 13 20:28:11.440953 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.440953 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8156e68-91ab-4c3b-a74e-6c8841219efe" containerName="manager" Jul 13 20:28:11.441058 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.441049 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8156e68-91ab-4c3b-a74e-6c8841219efe" containerName="manager" Jul 13 20:28:11.445585 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.445571 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d466f49f4-wgb26" Jul 13 20:28:11.450258 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.450234 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 20:28:11.450377 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.450267 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pxttn\"" Jul 13 20:28:11.450377 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.450241 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 20:28:11.460109 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.460085 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d466f49f4-wgb26"] Jul 13 20:28:11.517929 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.517891 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc59h\" (UniqueName: \"kubernetes.io/projected/28c12228-afd5-4a1d-8915-7b89239b332d-kube-api-access-xc59h\") pod \"maas-controller-d466f49f4-wgb26\" (UID: \"28c12228-afd5-4a1d-8915-7b89239b332d\") " pod="opendatahub/maas-controller-d466f49f4-wgb26" Jul 13 20:28:11.518078 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.517961 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28c12228-afd5-4a1d-8915-7b89239b332d-webhook-cert\") pod \"maas-controller-d466f49f4-wgb26\" (UID: \"28c12228-afd5-4a1d-8915-7b89239b332d\") " pod="opendatahub/maas-controller-d466f49f4-wgb26" Jul 13 20:28:11.619152 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.619117 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28c12228-afd5-4a1d-8915-7b89239b332d-webhook-cert\") pod \"maas-controller-d466f49f4-wgb26\" (UID: \"28c12228-afd5-4a1d-8915-7b89239b332d\") " pod="opendatahub/maas-controller-d466f49f4-wgb26" Jul 13 20:28:11.619301 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.619216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc59h\" (UniqueName: \"kubernetes.io/projected/28c12228-afd5-4a1d-8915-7b89239b332d-kube-api-access-xc59h\") pod \"maas-controller-d466f49f4-wgb26\" (UID: \"28c12228-afd5-4a1d-8915-7b89239b332d\") " pod="opendatahub/maas-controller-d466f49f4-wgb26" Jul 13 20:28:11.621540 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.621510 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28c12228-afd5-4a1d-8915-7b89239b332d-webhook-cert\") pod \"maas-controller-d466f49f4-wgb26\" (UID: \"28c12228-afd5-4a1d-8915-7b89239b332d\") " pod="opendatahub/maas-controller-d466f49f4-wgb26" Jul 13 20:28:11.630148 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.630121 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc59h\" (UniqueName: \"kubernetes.io/projected/28c12228-afd5-4a1d-8915-7b89239b332d-kube-api-access-xc59h\") pod \"maas-controller-d466f49f4-wgb26\" (UID: \"28c12228-afd5-4a1d-8915-7b89239b332d\") " pod="opendatahub/maas-controller-d466f49f4-wgb26" Jul 13 20:28:11.756235 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.756146 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d466f49f4-wgb26" Jul 13 20:28:11.893519 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:11.893488 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d466f49f4-wgb26"] Jul 13 20:28:11.894881 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:28:11.894858 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c12228_afd5_4a1d_8915_7b89239b332d.slice/crio-e9aec49251f265bd14b67487ba3c04d909f002a04bfea638765adc095a89fb9e WatchSource:0}: Error finding container e9aec49251f265bd14b67487ba3c04d909f002a04bfea638765adc095a89fb9e: Status 404 returned error can't find the container with id e9aec49251f265bd14b67487ba3c04d909f002a04bfea638765adc095a89fb9e Jul 13 20:28:12.642161 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:12.642076 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d466f49f4-wgb26" event={"ID":"28c12228-afd5-4a1d-8915-7b89239b332d","Type":"ContainerStarted","Data":"953ca80de4ad0483d1e9e5b63cebabaf6c065e450282febdb3c14c471c16ce8b"} Jul 13 20:28:12.642161 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:12.642137 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d466f49f4-wgb26" event={"ID":"28c12228-afd5-4a1d-8915-7b89239b332d","Type":"ContainerStarted","Data":"e9aec49251f265bd14b67487ba3c04d909f002a04bfea638765adc095a89fb9e"} Jul 13 20:28:12.642563 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:12.642204 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d466f49f4-wgb26" Jul 13 20:28:12.661656 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:12.661605 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d466f49f4-wgb26" podStartSLOduration=1.250375805 podStartE2EDuration="1.66158822s" totalImagesPullingTime="411.212415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:28:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:28:12.659009702 +0000 UTC m=+823.805276876" watchObservedRunningTime="2026-07-13 20:28:12.66158822 +0000 UTC m=+823.807855382" Jul 13 20:28:23.650902 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:28:23.650873 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d466f49f4-wgb26" Jul 13 20:29:29.343330 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:29:29.343293 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:29:29.343917 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:29:29.343619 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:32:54.487276 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:54.487239 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z"] Jul 13 20:32:54.488084 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:54.487459 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" podUID="6072ed02-479d-4fb5-ba92-0ae587c7d35a" containerName="manager" containerID="cri-o://5627859f8a9f620026d320d8b33dbbfb4ec8c997190ef08cb585578e3214fa1b" gracePeriod=10 Jul 13 20:32:58.152045 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:58.151954 2568 generic.go:356] "Generic (PLEG): container finished" podID="6072ed02-479d-4fb5-ba92-0ae587c7d35a" containerID="5627859f8a9f620026d320d8b33dbbfb4ec8c997190ef08cb585578e3214fa1b" exitCode=0 Jul 13 20:32:58.152045 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:58.152001 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" event={"ID":"6072ed02-479d-4fb5-ba92-0ae587c7d35a","Type":"ContainerDied","Data":"5627859f8a9f620026d320d8b33dbbfb4ec8c997190ef08cb585578e3214fa1b"} Jul 13 20:32:58.444722 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:58.444699 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:32:58.593375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:58.593343 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6072ed02-479d-4fb5-ba92-0ae587c7d35a-extensions-socket-volume\") pod \"6072ed02-479d-4fb5-ba92-0ae587c7d35a\" (UID: \"6072ed02-479d-4fb5-ba92-0ae587c7d35a\") " Jul 13 20:32:58.593563 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:58.593409 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsv8c\" (UniqueName: \"kubernetes.io/projected/6072ed02-479d-4fb5-ba92-0ae587c7d35a-kube-api-access-xsv8c\") pod \"6072ed02-479d-4fb5-ba92-0ae587c7d35a\" (UID: \"6072ed02-479d-4fb5-ba92-0ae587c7d35a\") " Jul 13 20:32:58.593750 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:58.593726 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6072ed02-479d-4fb5-ba92-0ae587c7d35a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6072ed02-479d-4fb5-ba92-0ae587c7d35a" (UID: "6072ed02-479d-4fb5-ba92-0ae587c7d35a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:32:58.595438 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:58.595398 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6072ed02-479d-4fb5-ba92-0ae587c7d35a-kube-api-access-xsv8c" (OuterVolumeSpecName: "kube-api-access-xsv8c") pod "6072ed02-479d-4fb5-ba92-0ae587c7d35a" (UID: "6072ed02-479d-4fb5-ba92-0ae587c7d35a"). InnerVolumeSpecName "kube-api-access-xsv8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:32:58.695048 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:58.694968 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6072ed02-479d-4fb5-ba92-0ae587c7d35a-extensions-socket-volume\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:32:58.695048 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:58.694993 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsv8c\" (UniqueName: \"kubernetes.io/projected/6072ed02-479d-4fb5-ba92-0ae587c7d35a-kube-api-access-xsv8c\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:32:59.166077 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:59.166049 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" Jul 13 20:32:59.166077 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:59.166068 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z" event={"ID":"6072ed02-479d-4fb5-ba92-0ae587c7d35a","Type":"ContainerDied","Data":"228a50fea7d01265bfb29bf02100b94060541125d911923e2c8dd8a9fd725cd0"} Jul 13 20:32:59.166777 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:59.166151 2568 scope.go:117] "RemoveContainer" containerID="5627859f8a9f620026d320d8b33dbbfb4ec8c997190ef08cb585578e3214fa1b" Jul 13 20:32:59.188325 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:59.188302 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z"] Jul 13 20:32:59.194278 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:59.194253 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mhq6z"] Jul 13 20:32:59.404267 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:32:59.404234 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6072ed02-479d-4fb5-ba92-0ae587c7d35a" path="/var/lib/kubelet/pods/6072ed02-479d-4fb5-ba92-0ae587c7d35a/volumes" Jul 13 20:33:55.629580 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.629505 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv"] Jul 13 20:33:55.630009 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.629946 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6072ed02-479d-4fb5-ba92-0ae587c7d35a" containerName="manager" Jul 13 20:33:55.630009 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.629956 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6072ed02-479d-4fb5-ba92-0ae587c7d35a" containerName="manager" Jul 13 20:33:55.630085 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.630036 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6072ed02-479d-4fb5-ba92-0ae587c7d35a" containerName="manager" Jul 13 20:33:55.633629 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.633609 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" Jul 13 20:33:55.637147 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.636809 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-42s9q\"" Jul 13 20:33:55.651825 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.651784 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv"] Jul 13 20:33:55.746878 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.746843 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqf2t\" (UniqueName: \"kubernetes.io/projected/57c8b1a6-aae9-40e7-94b1-c079ed126500-kube-api-access-zqf2t\") pod \"kuadrant-operator-controller-manager-86878c769f-tl2kv\" (UID: \"57c8b1a6-aae9-40e7-94b1-c079ed126500\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" Jul 13 20:33:55.746878 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.746879 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57c8b1a6-aae9-40e7-94b1-c079ed126500-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tl2kv\" (UID: \"57c8b1a6-aae9-40e7-94b1-c079ed126500\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" Jul 13 20:33:55.848189 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.848154 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqf2t\" (UniqueName: \"kubernetes.io/projected/57c8b1a6-aae9-40e7-94b1-c079ed126500-kube-api-access-zqf2t\") pod \"kuadrant-operator-controller-manager-86878c769f-tl2kv\" (UID: \"57c8b1a6-aae9-40e7-94b1-c079ed126500\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" Jul 13 20:33:55.848189 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.848192 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57c8b1a6-aae9-40e7-94b1-c079ed126500-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tl2kv\" (UID: \"57c8b1a6-aae9-40e7-94b1-c079ed126500\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" Jul 13 20:33:55.848603 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.848586 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/57c8b1a6-aae9-40e7-94b1-c079ed126500-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tl2kv\" (UID: \"57c8b1a6-aae9-40e7-94b1-c079ed126500\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" Jul 13 20:33:55.858775 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.858748 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqf2t\" (UniqueName: \"kubernetes.io/projected/57c8b1a6-aae9-40e7-94b1-c079ed126500-kube-api-access-zqf2t\") pod \"kuadrant-operator-controller-manager-86878c769f-tl2kv\" (UID: \"57c8b1a6-aae9-40e7-94b1-c079ed126500\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" Jul 13 20:33:55.954456 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:55.954371 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" Jul 13 20:33:56.095223 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:56.095197 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv"] Jul 13 20:33:56.097363 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:33:56.097333 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57c8b1a6_aae9_40e7_94b1_c079ed126500.slice/crio-46e2a4a0d719e7dfd8ea6af1e4079708bf1f16fe8da9d263b273a8d55b200018 WatchSource:0}: Error finding container 46e2a4a0d719e7dfd8ea6af1e4079708bf1f16fe8da9d263b273a8d55b200018: Status 404 returned error can't find the container with id 46e2a4a0d719e7dfd8ea6af1e4079708bf1f16fe8da9d263b273a8d55b200018 Jul 13 20:33:56.457196 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:56.457160 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" event={"ID":"57c8b1a6-aae9-40e7-94b1-c079ed126500","Type":"ContainerStarted","Data":"8d2657246aefaa37e2cc52349eae9ae22f6c1348907755a6563bdd2eea34ef88"} Jul 13 20:33:56.457362 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:56.457220 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" event={"ID":"57c8b1a6-aae9-40e7-94b1-c079ed126500","Type":"ContainerStarted","Data":"46e2a4a0d719e7dfd8ea6af1e4079708bf1f16fe8da9d263b273a8d55b200018"} Jul 13 20:33:56.457362 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:56.457252 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" Jul 13 20:33:56.479678 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:33:56.479628 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" podStartSLOduration=1.479611203 podStartE2EDuration="1.479611203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:33:56.479130036 +0000 UTC m=+1167.625397220" watchObservedRunningTime="2026-07-13 20:33:56.479611203 +0000 UTC m=+1167.625878365" Jul 13 20:34:07.464763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:34:07.464729 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tl2kv" Jul 13 20:34:29.378328 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:34:29.378301 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:34:29.379435 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:34:29.379390 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:39:29.415622 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:39:29.415587 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:39:29.418566 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:39:29.418534 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:40:22.845007 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:22.844973 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq"] Jul 13 20:40:22.848991 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:22.848968 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq" Jul 13 20:40:22.851509 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:22.851486 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-84mvh\"" Jul 13 20:40:22.857017 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:22.856993 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq"] Jul 13 20:40:22.947629 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:22.947593 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0501640-c850-43db-b25c-4f6e0c3fb447-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b435b208-x2wbq\" (UID: \"c0501640-c850-43db-b25c-4f6e0c3fb447\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq" Jul 13 20:40:23.048578 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:23.048543 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0501640-c850-43db-b25c-4f6e0c3fb447-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b435b208-x2wbq\" (UID: \"c0501640-c850-43db-b25c-4f6e0c3fb447\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq" Jul 13 20:40:23.049235 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:23.049211 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0501640-c850-43db-b25c-4f6e0c3fb447-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b435b208-x2wbq\" (UID: \"c0501640-c850-43db-b25c-4f6e0c3fb447\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq" Jul 13 20:40:23.165745 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:23.165658 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq" Jul 13 20:40:23.308715 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:23.308686 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq"] Jul 13 20:40:23.310341 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:40:23.310314 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0501640_c850_43db_b25c_4f6e0c3fb447.slice/crio-f2a012d37197ec4eeb299c945fcb7ac3d40e06e42422af2e2620048a5c04f7ff WatchSource:0}: Error finding container f2a012d37197ec4eeb299c945fcb7ac3d40e06e42422af2e2620048a5c04f7ff: Status 404 returned error can't find the container with id f2a012d37197ec4eeb299c945fcb7ac3d40e06e42422af2e2620048a5c04f7ff Jul 13 20:40:23.312183 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:23.312166 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:40:23.537637 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:23.537548 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq" event={"ID":"c0501640-c850-43db-b25c-4f6e0c3fb447","Type":"ContainerStarted","Data":"f2a012d37197ec4eeb299c945fcb7ac3d40e06e42422af2e2620048a5c04f7ff"} Jul 13 20:40:27.562675 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:27.562641 2568 generic.go:356] "Generic (PLEG): container finished" podID="c0501640-c850-43db-b25c-4f6e0c3fb447" containerID="8c377ad46c4effa404546a570edad1624b9577b8fccf495e05e8b0d8f7506d57" exitCode=7 Jul 13 20:40:27.563038 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:27.562700 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq" event={"ID":"c0501640-c850-43db-b25c-4f6e0c3fb447","Type":"ContainerDied","Data":"8c377ad46c4effa404546a570edad1624b9577b8fccf495e05e8b0d8f7506d57"} Jul 13 20:40:27.563038 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:27.562892 2568 scope.go:117] "RemoveContainer" containerID="8c377ad46c4effa404546a570edad1624b9577b8fccf495e05e8b0d8f7506d57" Jul 13 20:40:28.570818 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:28.570785 2568 generic.go:356] "Generic (PLEG): container finished" podID="c0501640-c850-43db-b25c-4f6e0c3fb447" containerID="beaea399a425a8dea1ab376ae0b7440d2f6248de8ff7665259e56e272171e1ac" exitCode=0 Jul 13 20:40:28.571227 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:28.570871 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq" event={"ID":"c0501640-c850-43db-b25c-4f6e0c3fb447","Type":"ContainerDied","Data":"beaea399a425a8dea1ab376ae0b7440d2f6248de8ff7665259e56e272171e1ac"} Jul 13 20:40:28.571227 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:28.570935 2568 scope.go:117] "RemoveContainer" containerID="8c377ad46c4effa404546a570edad1624b9577b8fccf495e05e8b0d8f7506d57" Jul 13 20:40:29.717678 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:29.717655 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq" Jul 13 20:40:29.814898 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:29.814871 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0501640-c850-43db-b25c-4f6e0c3fb447-maas-api-service-ca\") pod \"c0501640-c850-43db-b25c-4f6e0c3fb447\" (UID: \"c0501640-c850-43db-b25c-4f6e0c3fb447\") " Jul 13 20:40:29.815320 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:29.815292 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0501640-c850-43db-b25c-4f6e0c3fb447-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c0501640-c850-43db-b25c-4f6e0c3fb447" (UID: "c0501640-c850-43db-b25c-4f6e0c3fb447"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:40:29.916222 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:29.916186 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0501640-c850-43db-b25c-4f6e0c3fb447-maas-api-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:40:30.584818 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:30.584794 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq" Jul 13 20:40:30.584981 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:30.584828 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b435b208-x2wbq" event={"ID":"c0501640-c850-43db-b25c-4f6e0c3fb447","Type":"ContainerDied","Data":"f2a012d37197ec4eeb299c945fcb7ac3d40e06e42422af2e2620048a5c04f7ff"} Jul 13 20:40:30.584981 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:40:30.584877 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2a012d37197ec4eeb299c945fcb7ac3d40e06e42422af2e2620048a5c04f7ff" Jul 13 20:41:44.746925 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:44.746892 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb"] Jul 13 20:41:44.747376 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:44.747336 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0501640-c850-43db-b25c-4f6e0c3fb447" containerName="revoke-keys" Jul 13 20:41:44.747376 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:44.747346 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0501640-c850-43db-b25c-4f6e0c3fb447" containerName="revoke-keys" Jul 13 20:41:44.747476 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:44.747447 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0501640-c850-43db-b25c-4f6e0c3fb447" containerName="revoke-keys" Jul 13 20:41:44.747476 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:44.747456 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0501640-c850-43db-b25c-4f6e0c3fb447" containerName="revoke-keys" Jul 13 20:41:44.750846 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:44.750829 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" Jul 13 20:41:44.753323 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:44.753301 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-84mvh\"" Jul 13 20:41:44.759912 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:44.759887 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb"] Jul 13 20:41:44.850091 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:44.850053 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bc682d-f1e1-4c44-8e90-634b07dadde5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb\" (UID: \"e8bc682d-f1e1-4c44-8e90-634b07dadde5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" Jul 13 20:41:44.950722 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:44.950690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bc682d-f1e1-4c44-8e90-634b07dadde5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb\" (UID: \"e8bc682d-f1e1-4c44-8e90-634b07dadde5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" Jul 13 20:41:44.951257 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:44.951240 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bc682d-f1e1-4c44-8e90-634b07dadde5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb\" (UID: \"e8bc682d-f1e1-4c44-8e90-634b07dadde5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" Jul 13 20:41:45.062678 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:45.062645 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" Jul 13 20:41:45.403056 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:41:45.403020 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8bc682d_f1e1_4c44_8e90_634b07dadde5.slice/crio-717f5b2596c347f9bb3badc545c042b7e966f0bccfd2b25a96ccba810501bd24 WatchSource:0}: Error finding container 717f5b2596c347f9bb3badc545c042b7e966f0bccfd2b25a96ccba810501bd24: Status 404 returned error can't find the container with id 717f5b2596c347f9bb3badc545c042b7e966f0bccfd2b25a96ccba810501bd24 Jul 13 20:41:45.403747 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:45.403649 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb"] Jul 13 20:41:45.981295 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:45.981206 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" event={"ID":"e8bc682d-f1e1-4c44-8e90-634b07dadde5","Type":"ContainerStarted","Data":"29b38b99e6195c673ea91b138c10bdbc2e37c16537641513418d343341434dd9"} Jul 13 20:41:45.981295 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:45.981254 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" event={"ID":"e8bc682d-f1e1-4c44-8e90-634b07dadde5","Type":"ContainerStarted","Data":"717f5b2596c347f9bb3badc545c042b7e966f0bccfd2b25a96ccba810501bd24"} Jul 13 20:41:46.000261 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:46.000216 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" podStartSLOduration=2.000201616 podStartE2EDuration="2.000201616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:41:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:41:45.997354429 +0000 UTC m=+1637.143621593" watchObservedRunningTime="2026-07-13 20:41:46.000201616 +0000 UTC m=+1637.146468776" Jul 13 20:41:46.987836 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:46.987802 2568 generic.go:356] "Generic (PLEG): container finished" podID="e8bc682d-f1e1-4c44-8e90-634b07dadde5" containerID="29b38b99e6195c673ea91b138c10bdbc2e37c16537641513418d343341434dd9" exitCode=0 Jul 13 20:41:46.988215 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:46.987886 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" event={"ID":"e8bc682d-f1e1-4c44-8e90-634b07dadde5","Type":"ContainerDied","Data":"29b38b99e6195c673ea91b138c10bdbc2e37c16537641513418d343341434dd9"} Jul 13 20:41:48.134005 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:48.133982 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" Jul 13 20:41:48.285208 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:48.285130 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bc682d-f1e1-4c44-8e90-634b07dadde5-maas-api-service-ca\") pod \"e8bc682d-f1e1-4c44-8e90-634b07dadde5\" (UID: \"e8bc682d-f1e1-4c44-8e90-634b07dadde5\") " Jul 13 20:41:48.285492 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:48.285470 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8bc682d-f1e1-4c44-8e90-634b07dadde5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8bc682d-f1e1-4c44-8e90-634b07dadde5" (UID: "e8bc682d-f1e1-4c44-8e90-634b07dadde5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:41:48.386584 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:48.386549 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bc682d-f1e1-4c44-8e90-634b07dadde5-maas-api-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:41:49.000641 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:49.000604 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" Jul 13 20:41:49.000816 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:49.000608 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb" event={"ID":"e8bc682d-f1e1-4c44-8e90-634b07dadde5","Type":"ContainerDied","Data":"717f5b2596c347f9bb3badc545c042b7e966f0bccfd2b25a96ccba810501bd24"} Jul 13 20:41:49.000816 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:41:49.000760 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="717f5b2596c347f9bb3badc545c042b7e966f0bccfd2b25a96ccba810501bd24" Jul 13 20:44:29.449788 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:44:29.449677 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:44:29.457361 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:44:29.453277 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:45:32.039901 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.039866 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs"] Jul 13 20:45:32.040575 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.040557 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0501640-c850-43db-b25c-4f6e0c3fb447" containerName="revoke-keys" Jul 13 20:45:32.040635 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.040579 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0501640-c850-43db-b25c-4f6e0c3fb447" containerName="revoke-keys" Jul 13 20:45:32.040727 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.040714 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8bc682d-f1e1-4c44-8e90-634b07dadde5" containerName="revoke-keys" Jul 13 20:45:32.040763 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.040728 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8bc682d-f1e1-4c44-8e90-634b07dadde5" containerName="revoke-keys" Jul 13 20:45:32.040896 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.040886 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8bc682d-f1e1-4c44-8e90-634b07dadde5" containerName="revoke-keys" Jul 13 20:45:32.044626 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.044606 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" Jul 13 20:45:32.046908 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.046882 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-84mvh\"" Jul 13 20:45:32.050358 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.050336 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs"] Jul 13 20:45:32.110816 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.110783 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33dd272f-7f51-42fd-9848-97920b412bc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs\" (UID: \"33dd272f-7f51-42fd-9848-97920b412bc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" Jul 13 20:45:32.211756 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.211726 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33dd272f-7f51-42fd-9848-97920b412bc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs\" (UID: \"33dd272f-7f51-42fd-9848-97920b412bc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" Jul 13 20:45:32.212383 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.212363 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33dd272f-7f51-42fd-9848-97920b412bc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs\" (UID: \"33dd272f-7f51-42fd-9848-97920b412bc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" Jul 13 20:45:32.357869 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.357833 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" Jul 13 20:45:32.497368 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:32.497331 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs"] Jul 13 20:45:32.498588 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:45:32.498556 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33dd272f_7f51_42fd_9848_97920b412bc7.slice/crio-f3c9cf66b9aac759a46df01382d0693c628006ecd03e74fe10336a1493eb1fed WatchSource:0}: Error finding container f3c9cf66b9aac759a46df01382d0693c628006ecd03e74fe10336a1493eb1fed: Status 404 returned error can't find the container with id f3c9cf66b9aac759a46df01382d0693c628006ecd03e74fe10336a1493eb1fed Jul 13 20:45:33.222121 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:33.222089 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" event={"ID":"33dd272f-7f51-42fd-9848-97920b412bc7","Type":"ContainerStarted","Data":"46635ed599ae44d344aa8b7fc6ada909217068d14b8ddc88b8925a973773bb81"} Jul 13 20:45:33.222552 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:33.222143 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" event={"ID":"33dd272f-7f51-42fd-9848-97920b412bc7","Type":"ContainerStarted","Data":"f3c9cf66b9aac759a46df01382d0693c628006ecd03e74fe10336a1493eb1fed"} Jul 13 20:45:33.236484 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:33.236433 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" podStartSLOduration=1.23639652 podStartE2EDuration="1.23639652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:45:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:45:33.235337753 +0000 UTC m=+1864.381604915" watchObservedRunningTime="2026-07-13 20:45:33.23639652 +0000 UTC m=+1864.382663685" Jul 13 20:45:34.229198 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:34.229161 2568 generic.go:356] "Generic (PLEG): container finished" podID="33dd272f-7f51-42fd-9848-97920b412bc7" containerID="46635ed599ae44d344aa8b7fc6ada909217068d14b8ddc88b8925a973773bb81" exitCode=0 Jul 13 20:45:34.229604 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:34.229244 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" event={"ID":"33dd272f-7f51-42fd-9848-97920b412bc7","Type":"ContainerDied","Data":"46635ed599ae44d344aa8b7fc6ada909217068d14b8ddc88b8925a973773bb81"} Jul 13 20:45:35.381046 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:35.381022 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" Jul 13 20:45:35.445759 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:35.445732 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33dd272f-7f51-42fd-9848-97920b412bc7-maas-api-service-ca\") pod \"33dd272f-7f51-42fd-9848-97920b412bc7\" (UID: \"33dd272f-7f51-42fd-9848-97920b412bc7\") " Jul 13 20:45:35.446131 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:35.446104 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33dd272f-7f51-42fd-9848-97920b412bc7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "33dd272f-7f51-42fd-9848-97920b412bc7" (UID: "33dd272f-7f51-42fd-9848-97920b412bc7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:45:35.546711 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:35.546683 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33dd272f-7f51-42fd-9848-97920b412bc7-maas-api-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:45:36.244526 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:36.244499 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" Jul 13 20:45:36.244727 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:36.244498 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs" event={"ID":"33dd272f-7f51-42fd-9848-97920b412bc7","Type":"ContainerDied","Data":"f3c9cf66b9aac759a46df01382d0693c628006ecd03e74fe10336a1493eb1fed"} Jul 13 20:45:36.244727 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:45:36.244623 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3c9cf66b9aac759a46df01382d0693c628006ecd03e74fe10336a1493eb1fed" Jul 13 20:47:08.593117 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:08.593085 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln"] Jul 13 20:47:08.593614 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:08.593577 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33dd272f-7f51-42fd-9848-97920b412bc7" containerName="revoke-keys" Jul 13 20:47:08.593614 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:08.593589 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="33dd272f-7f51-42fd-9848-97920b412bc7" containerName="revoke-keys" Jul 13 20:47:08.593694 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:08.593657 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="33dd272f-7f51-42fd-9848-97920b412bc7" containerName="revoke-keys" Jul 13 20:47:08.596731 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:08.596716 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" Jul 13 20:47:08.599239 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:08.599223 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-84mvh\"" Jul 13 20:47:08.603186 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:08.603163 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln"] Jul 13 20:47:08.685765 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:08.685732 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b99cf1-f8db-4f0a-9308-1b7bda8492e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d387c72d-h69ln\" (UID: \"29b99cf1-f8db-4f0a-9308-1b7bda8492e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" Jul 13 20:47:08.786310 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:08.786280 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b99cf1-f8db-4f0a-9308-1b7bda8492e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d387c72d-h69ln\" (UID: \"29b99cf1-f8db-4f0a-9308-1b7bda8492e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" Jul 13 20:47:08.786950 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:08.786925 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b99cf1-f8db-4f0a-9308-1b7bda8492e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d387c72d-h69ln\" (UID: \"29b99cf1-f8db-4f0a-9308-1b7bda8492e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" Jul 13 20:47:08.909191 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:08.909096 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" Jul 13 20:47:09.057741 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:09.057705 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln"] Jul 13 20:47:09.062333 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:47:09.062304 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b99cf1_f8db_4f0a_9308_1b7bda8492e7.slice/crio-4fcf1c2af60293ab75e1ca10475be1730e939e9778598e5921190b67ce403150 WatchSource:0}: Error finding container 4fcf1c2af60293ab75e1ca10475be1730e939e9778598e5921190b67ce403150: Status 404 returned error can't find the container with id 4fcf1c2af60293ab75e1ca10475be1730e939e9778598e5921190b67ce403150 Jul 13 20:47:09.747740 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:09.747696 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" event={"ID":"29b99cf1-f8db-4f0a-9308-1b7bda8492e7","Type":"ContainerStarted","Data":"0d118e3071c6caf2522d7be56b0bd48de66067eb3d4ee2ef57ee35ef1086cd52"} Jul 13 20:47:09.748213 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:09.747766 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" event={"ID":"29b99cf1-f8db-4f0a-9308-1b7bda8492e7","Type":"ContainerStarted","Data":"4fcf1c2af60293ab75e1ca10475be1730e939e9778598e5921190b67ce403150"} Jul 13 20:47:09.763363 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:09.763314 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" podStartSLOduration=1.763294677 podStartE2EDuration="1.763294677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:47:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:47:09.761236376 +0000 UTC m=+1960.907503538" watchObservedRunningTime="2026-07-13 20:47:09.763294677 +0000 UTC m=+1960.909561839" Jul 13 20:47:10.754786 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:10.754748 2568 generic.go:356] "Generic (PLEG): container finished" podID="29b99cf1-f8db-4f0a-9308-1b7bda8492e7" containerID="0d118e3071c6caf2522d7be56b0bd48de66067eb3d4ee2ef57ee35ef1086cd52" exitCode=0 Jul 13 20:47:10.755184 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:10.754798 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" event={"ID":"29b99cf1-f8db-4f0a-9308-1b7bda8492e7","Type":"ContainerDied","Data":"0d118e3071c6caf2522d7be56b0bd48de66067eb3d4ee2ef57ee35ef1086cd52"} Jul 13 20:47:11.898882 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:11.898859 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" Jul 13 20:47:11.916354 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:11.916326 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b99cf1-f8db-4f0a-9308-1b7bda8492e7-maas-api-service-ca\") pod \"29b99cf1-f8db-4f0a-9308-1b7bda8492e7\" (UID: \"29b99cf1-f8db-4f0a-9308-1b7bda8492e7\") " Jul 13 20:47:11.916707 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:11.916686 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29b99cf1-f8db-4f0a-9308-1b7bda8492e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29b99cf1-f8db-4f0a-9308-1b7bda8492e7" (UID: "29b99cf1-f8db-4f0a-9308-1b7bda8492e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:47:12.017831 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:12.017750 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b99cf1-f8db-4f0a-9308-1b7bda8492e7-maas-api-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:47:12.766536 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:12.766506 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" Jul 13 20:47:12.766713 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:12.766544 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d387c72d-h69ln" event={"ID":"29b99cf1-f8db-4f0a-9308-1b7bda8492e7","Type":"ContainerDied","Data":"4fcf1c2af60293ab75e1ca10475be1730e939e9778598e5921190b67ce403150"} Jul 13 20:47:12.766713 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:47:12.766594 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fcf1c2af60293ab75e1ca10475be1730e939e9778598e5921190b67ce403150" Jul 13 20:49:29.486831 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:49:29.486730 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:49:29.491827 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:49:29.491808 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:50:56.419447 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.419401 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79"] Jul 13 20:50:56.419935 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.419921 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29b99cf1-f8db-4f0a-9308-1b7bda8492e7" containerName="revoke-keys" Jul 13 20:50:56.419981 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.419937 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="29b99cf1-f8db-4f0a-9308-1b7bda8492e7" containerName="revoke-keys" Jul 13 20:50:56.420032 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.420022 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="29b99cf1-f8db-4f0a-9308-1b7bda8492e7" containerName="revoke-keys" Jul 13 20:50:56.423327 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.423310 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79" Jul 13 20:50:56.425723 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.425701 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-84mvh\"" Jul 13 20:50:56.428772 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.428745 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79"] Jul 13 20:50:56.464949 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.464921 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7818c224-90ef-4930-8adb-0be7e81f4de0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79\" (UID: \"7818c224-90ef-4930-8adb-0be7e81f4de0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79" Jul 13 20:50:56.566072 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.566042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7818c224-90ef-4930-8adb-0be7e81f4de0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79\" (UID: \"7818c224-90ef-4930-8adb-0be7e81f4de0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79" Jul 13 20:50:56.566753 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.566730 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7818c224-90ef-4930-8adb-0be7e81f4de0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79\" (UID: \"7818c224-90ef-4930-8adb-0be7e81f4de0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79" Jul 13 20:50:56.736062 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.735954 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79" Jul 13 20:50:56.879747 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:56.879721 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79"] Jul 13 20:50:56.881822 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:50:56.881793 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7818c224_90ef_4930_8adb_0be7e81f4de0.slice/crio-a30ecf09d85bfe61b78ff454b87de01b346e02f515ac1f90fe938c218d0a6d69 WatchSource:0}: Error finding container a30ecf09d85bfe61b78ff454b87de01b346e02f515ac1f90fe938c218d0a6d69: Status 404 returned error can't find the container with id a30ecf09d85bfe61b78ff454b87de01b346e02f515ac1f90fe938c218d0a6d69 Jul 13 20:50:57.009791 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:57.009705 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79" event={"ID":"7818c224-90ef-4930-8adb-0be7e81f4de0","Type":"ContainerStarted","Data":"a30ecf09d85bfe61b78ff454b87de01b346e02f515ac1f90fe938c218d0a6d69"} Jul 13 20:50:58.017862 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:58.017771 2568 generic.go:356] "Generic (PLEG): container finished" podID="7818c224-90ef-4930-8adb-0be7e81f4de0" containerID="bd7a01f9e166faddf466633ee308f67cf298f7dd3fa9cd856f044a52f2c39f38" exitCode=0 Jul 13 20:50:58.017862 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:58.017854 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79" event={"ID":"7818c224-90ef-4930-8adb-0be7e81f4de0","Type":"ContainerDied","Data":"bd7a01f9e166faddf466633ee308f67cf298f7dd3fa9cd856f044a52f2c39f38"} Jul 13 20:50:59.162454 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:59.162408 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79" Jul 13 20:50:59.294461 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:59.294435 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7818c224-90ef-4930-8adb-0be7e81f4de0-maas-api-service-ca\") pod \"7818c224-90ef-4930-8adb-0be7e81f4de0\" (UID: \"7818c224-90ef-4930-8adb-0be7e81f4de0\") " Jul 13 20:50:59.294774 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:59.294753 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7818c224-90ef-4930-8adb-0be7e81f4de0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7818c224-90ef-4930-8adb-0be7e81f4de0" (UID: "7818c224-90ef-4930-8adb-0be7e81f4de0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:50:59.395408 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:50:59.395375 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7818c224-90ef-4930-8adb-0be7e81f4de0-maas-api-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:51:00.030135 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:00.030107 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79" Jul 13 20:51:00.030315 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:00.030140 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79" event={"ID":"7818c224-90ef-4930-8adb-0be7e81f4de0","Type":"ContainerDied","Data":"a30ecf09d85bfe61b78ff454b87de01b346e02f515ac1f90fe938c218d0a6d69"} Jul 13 20:51:00.030315 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:00.030204 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a30ecf09d85bfe61b78ff454b87de01b346e02f515ac1f90fe938c218d0a6d69" Jul 13 20:51:22.579404 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:22.579369 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd"] Jul 13 20:51:22.579881 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:22.579832 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7818c224-90ef-4930-8adb-0be7e81f4de0" containerName="revoke-keys" Jul 13 20:51:22.579881 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:22.579843 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7818c224-90ef-4930-8adb-0be7e81f4de0" containerName="revoke-keys" Jul 13 20:51:22.579970 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:22.579916 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7818c224-90ef-4930-8adb-0be7e81f4de0" containerName="revoke-keys" Jul 13 20:51:22.583198 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:22.583180 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd" Jul 13 20:51:22.585524 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:22.585506 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-84mvh\"" Jul 13 20:51:22.589729 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:22.589694 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd"] Jul 13 20:51:22.729312 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:22.729282 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc87d7e3-ea44-4bbf-a537-3ce888f9af06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd\" (UID: \"cc87d7e3-ea44-4bbf-a537-3ce888f9af06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd" Jul 13 20:51:22.830640 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:22.830555 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc87d7e3-ea44-4bbf-a537-3ce888f9af06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd\" (UID: \"cc87d7e3-ea44-4bbf-a537-3ce888f9af06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd" Jul 13 20:51:22.831220 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:22.831201 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc87d7e3-ea44-4bbf-a537-3ce888f9af06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd\" (UID: \"cc87d7e3-ea44-4bbf-a537-3ce888f9af06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd" Jul 13 20:51:22.896580 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:22.896548 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd" Jul 13 20:51:23.035366 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:23.035338 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd"] Jul 13 20:51:23.035745 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:51:23.035723 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc87d7e3_ea44_4bbf_a537_3ce888f9af06.slice/crio-df101aa548853ccafb55ade7cf121180260f7a8c59dd1ab911501fd6c8ff3483 WatchSource:0}: Error finding container df101aa548853ccafb55ade7cf121180260f7a8c59dd1ab911501fd6c8ff3483: Status 404 returned error can't find the container with id df101aa548853ccafb55ade7cf121180260f7a8c59dd1ab911501fd6c8ff3483 Jul 13 20:51:23.161405 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:23.161311 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd" event={"ID":"cc87d7e3-ea44-4bbf-a537-3ce888f9af06","Type":"ContainerStarted","Data":"df101aa548853ccafb55ade7cf121180260f7a8c59dd1ab911501fd6c8ff3483"} Jul 13 20:51:24.171006 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:24.170969 2568 generic.go:356] "Generic (PLEG): container finished" podID="cc87d7e3-ea44-4bbf-a537-3ce888f9af06" containerID="702f3e56b6c644e3b07d1a8ac54656ba35f3c7da54f3a40e07bf41dfeac2f512" exitCode=0 Jul 13 20:51:24.171393 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:24.171055 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd" event={"ID":"cc87d7e3-ea44-4bbf-a537-3ce888f9af06","Type":"ContainerDied","Data":"702f3e56b6c644e3b07d1a8ac54656ba35f3c7da54f3a40e07bf41dfeac2f512"} Jul 13 20:51:25.310284 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:25.310261 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd" Jul 13 20:51:25.350864 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:25.350834 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc87d7e3-ea44-4bbf-a537-3ce888f9af06-maas-api-service-ca\") pod \"cc87d7e3-ea44-4bbf-a537-3ce888f9af06\" (UID: \"cc87d7e3-ea44-4bbf-a537-3ce888f9af06\") " Jul 13 20:51:25.351222 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:25.351198 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc87d7e3-ea44-4bbf-a537-3ce888f9af06-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cc87d7e3-ea44-4bbf-a537-3ce888f9af06" (UID: "cc87d7e3-ea44-4bbf-a537-3ce888f9af06"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:51:25.455487 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:25.452347 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc87d7e3-ea44-4bbf-a537-3ce888f9af06-maas-api-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:51:26.183790 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:26.183754 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd" Jul 13 20:51:26.183983 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:26.183789 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd" event={"ID":"cc87d7e3-ea44-4bbf-a537-3ce888f9af06","Type":"ContainerDied","Data":"df101aa548853ccafb55ade7cf121180260f7a8c59dd1ab911501fd6c8ff3483"} Jul 13 20:51:26.183983 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:51:26.183871 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df101aa548853ccafb55ade7cf121180260f7a8c59dd1ab911501fd6c8ff3483" Jul 13 20:52:13.216807 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.216724 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48"] Jul 13 20:52:13.217255 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.217217 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc87d7e3-ea44-4bbf-a537-3ce888f9af06" containerName="revoke-keys" Jul 13 20:52:13.217255 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.217231 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc87d7e3-ea44-4bbf-a537-3ce888f9af06" containerName="revoke-keys" Jul 13 20:52:13.217330 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.217316 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc87d7e3-ea44-4bbf-a537-3ce888f9af06" containerName="revoke-keys" Jul 13 20:52:13.220521 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.220500 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48" Jul 13 20:52:13.222846 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.222825 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-84mvh\"" Jul 13 20:52:13.226479 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.226457 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48"] Jul 13 20:52:13.342126 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.342096 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7477a21-dc26-4e02-a453-b8e1802f8d39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ad7818-9lh48\" (UID: \"e7477a21-dc26-4e02-a453-b8e1802f8d39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48" Jul 13 20:52:13.443285 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.443247 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7477a21-dc26-4e02-a453-b8e1802f8d39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ad7818-9lh48\" (UID: \"e7477a21-dc26-4e02-a453-b8e1802f8d39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48" Jul 13 20:52:13.443913 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.443893 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7477a21-dc26-4e02-a453-b8e1802f8d39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ad7818-9lh48\" (UID: \"e7477a21-dc26-4e02-a453-b8e1802f8d39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48" Jul 13 20:52:13.532524 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.532443 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48" Jul 13 20:52:13.672224 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:13.672194 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48"] Jul 13 20:52:13.674165 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:52:13.674132 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7477a21_dc26_4e02_a453_b8e1802f8d39.slice/crio-ac3ad96e8afb57cee48f23c00f02fc8b2374bd56a4163273aaddaee9542d12fd WatchSource:0}: Error finding container ac3ad96e8afb57cee48f23c00f02fc8b2374bd56a4163273aaddaee9542d12fd: Status 404 returned error can't find the container with id ac3ad96e8afb57cee48f23c00f02fc8b2374bd56a4163273aaddaee9542d12fd Jul 13 20:52:14.462650 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:14.462618 2568 generic.go:356] "Generic (PLEG): container finished" podID="e7477a21-dc26-4e02-a453-b8e1802f8d39" containerID="53186372d0122373e3f0557f18454ce401b8ac867eb04614264a68aac52935b4" exitCode=0 Jul 13 20:52:14.463074 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:14.462706 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48" event={"ID":"e7477a21-dc26-4e02-a453-b8e1802f8d39","Type":"ContainerDied","Data":"53186372d0122373e3f0557f18454ce401b8ac867eb04614264a68aac52935b4"} Jul 13 20:52:14.463074 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:14.462757 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48" event={"ID":"e7477a21-dc26-4e02-a453-b8e1802f8d39","Type":"ContainerStarted","Data":"ac3ad96e8afb57cee48f23c00f02fc8b2374bd56a4163273aaddaee9542d12fd"} Jul 13 20:52:15.603468 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:15.603446 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48" Jul 13 20:52:15.769242 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:15.769153 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7477a21-dc26-4e02-a453-b8e1802f8d39-maas-api-service-ca\") pod \"e7477a21-dc26-4e02-a453-b8e1802f8d39\" (UID: \"e7477a21-dc26-4e02-a453-b8e1802f8d39\") " Jul 13 20:52:15.769599 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:15.769573 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7477a21-dc26-4e02-a453-b8e1802f8d39-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e7477a21-dc26-4e02-a453-b8e1802f8d39" (UID: "e7477a21-dc26-4e02-a453-b8e1802f8d39"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:52:15.870673 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:15.870637 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7477a21-dc26-4e02-a453-b8e1802f8d39-maas-api-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:52:16.474776 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:16.474740 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48" event={"ID":"e7477a21-dc26-4e02-a453-b8e1802f8d39","Type":"ContainerDied","Data":"ac3ad96e8afb57cee48f23c00f02fc8b2374bd56a4163273aaddaee9542d12fd"} Jul 13 20:52:16.474951 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:16.474792 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac3ad96e8afb57cee48f23c00f02fc8b2374bd56a4163273aaddaee9542d12fd" Jul 13 20:52:16.474951 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:52:16.474748 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ad7818-9lh48" Jul 13 20:54:29.523303 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:54:29.523186 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:54:29.529254 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:54:29.529236 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:59:09.390032 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.389991 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7326-140510219594800"] Jul 13 20:59:09.390684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.390550 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7477a21-dc26-4e02-a453-b8e1802f8d39" containerName="revoke-keys" Jul 13 20:59:09.390684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.390564 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7477a21-dc26-4e02-a453-b8e1802f8d39" containerName="revoke-keys" Jul 13 20:59:09.390684 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.390638 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7477a21-dc26-4e02-a453-b8e1802f8d39" containerName="revoke-keys" Jul 13 20:59:09.399342 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.399306 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510219594800" Jul 13 20:59:09.402509 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.402468 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-kxvn6\"" Jul 13 20:59:09.413089 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.413064 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7326-140510219594800"] Jul 13 20:59:09.502390 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.502352 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sldv6\" (UniqueName: \"kubernetes.io/projected/c5886fa4-5da2-4f2b-9cbc-5b8927b903c6-kube-api-access-sldv6\") pod \"test-curl-7326-140510219594800\" (UID: \"c5886fa4-5da2-4f2b-9cbc-5b8927b903c6\") " pod="openshift-ingress/test-curl-7326-140510219594800" Jul 13 20:59:09.603702 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.603655 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sldv6\" (UniqueName: \"kubernetes.io/projected/c5886fa4-5da2-4f2b-9cbc-5b8927b903c6-kube-api-access-sldv6\") pod \"test-curl-7326-140510219594800\" (UID: \"c5886fa4-5da2-4f2b-9cbc-5b8927b903c6\") " pod="openshift-ingress/test-curl-7326-140510219594800" Jul 13 20:59:09.611377 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.611345 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sldv6\" (UniqueName: \"kubernetes.io/projected/c5886fa4-5da2-4f2b-9cbc-5b8927b903c6-kube-api-access-sldv6\") pod \"test-curl-7326-140510219594800\" (UID: \"c5886fa4-5da2-4f2b-9cbc-5b8927b903c6\") " pod="openshift-ingress/test-curl-7326-140510219594800" Jul 13 20:59:09.716187 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.716093 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510219594800" Jul 13 20:59:09.851882 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.851858 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7326-140510219594800"] Jul 13 20:59:09.853838 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:59:09.853812 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5886fa4_5da2_4f2b_9cbc_5b8927b903c6.slice/crio-fe04a11df946eef235d48c8e913eb061860c0e38d9d7ee044ada70054cd22ccd WatchSource:0}: Error finding container fe04a11df946eef235d48c8e913eb061860c0e38d9d7ee044ada70054cd22ccd: Status 404 returned error can't find the container with id fe04a11df946eef235d48c8e913eb061860c0e38d9d7ee044ada70054cd22ccd Jul 13 20:59:09.855092 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:09.855076 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:59:10.819001 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:10.818914 2568 generic.go:356] "Generic (PLEG): container finished" podID="c5886fa4-5da2-4f2b-9cbc-5b8927b903c6" containerID="b794040d29eb1d295946a1c9c4b2bea56c06e91aa06eceda1504ca17aa5d5adf" exitCode=0 Jul 13 20:59:10.819359 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:10.819001 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7326-140510219594800" event={"ID":"c5886fa4-5da2-4f2b-9cbc-5b8927b903c6","Type":"ContainerDied","Data":"b794040d29eb1d295946a1c9c4b2bea56c06e91aa06eceda1504ca17aa5d5adf"} Jul 13 20:59:10.819359 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:10.819046 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7326-140510219594800" event={"ID":"c5886fa4-5da2-4f2b-9cbc-5b8927b903c6","Type":"ContainerStarted","Data":"fe04a11df946eef235d48c8e913eb061860c0e38d9d7ee044ada70054cd22ccd"} Jul 13 20:59:11.978285 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:11.978260 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510219594800" Jul 13 20:59:12.010504 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:12.010479 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7326-140510219594800_c5886fa4-5da2-4f2b-9cbc-5b8927b903c6/test-curl-7326-140510219594800/0.log" Jul 13 20:59:12.029627 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:12.029602 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sldv6\" (UniqueName: \"kubernetes.io/projected/c5886fa4-5da2-4f2b-9cbc-5b8927b903c6-kube-api-access-sldv6\") pod \"c5886fa4-5da2-4f2b-9cbc-5b8927b903c6\" (UID: \"c5886fa4-5da2-4f2b-9cbc-5b8927b903c6\") " Jul 13 20:59:12.032211 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:12.031994 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5886fa4-5da2-4f2b-9cbc-5b8927b903c6-kube-api-access-sldv6" (OuterVolumeSpecName: "kube-api-access-sldv6") pod "c5886fa4-5da2-4f2b-9cbc-5b8927b903c6" (UID: "c5886fa4-5da2-4f2b-9cbc-5b8927b903c6"). InnerVolumeSpecName "kube-api-access-sldv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:59:12.033673 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:12.033651 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7326-140510219594800"] Jul 13 20:59:12.035883 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:12.035858 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7326-140510219594800"] Jul 13 20:59:12.130830 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:12.130729 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sldv6\" (UniqueName: \"kubernetes.io/projected/c5886fa4-5da2-4f2b-9cbc-5b8927b903c6-kube-api-access-sldv6\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:59:12.831841 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:12.831807 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510219594800" Jul 13 20:59:12.831841 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:12.831845 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe04a11df946eef235d48c8e913eb061860c0e38d9d7ee044ada70054cd22ccd" Jul 13 20:59:12.841468 ip-10-0-135-149 kubenswrapper[2568]: E0713 20:59:12.841442 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7326-140510219594800\" is forbidden: User \"system:node:ip-10-0-135-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-149.ec2.internal' and this object" podUID="c5886fa4-5da2-4f2b-9cbc-5b8927b903c6" pod="openshift-ingress/test-curl-7326-140510219594800" Jul 13 20:59:13.403601 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:13.403566 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5886fa4-5da2-4f2b-9cbc-5b8927b903c6" path="/var/lib/kubelet/pods/c5886fa4-5da2-4f2b-9cbc-5b8927b903c6/volumes" Jul 13 20:59:14.769065 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:14.769030 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7326-140510225509232"] Jul 13 20:59:14.769678 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:14.769658 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5886fa4-5da2-4f2b-9cbc-5b8927b903c6" containerName="test-curl-7326-140510219594800" Jul 13 20:59:14.769734 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:14.769681 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5886fa4-5da2-4f2b-9cbc-5b8927b903c6" containerName="test-curl-7326-140510219594800" Jul 13 20:59:14.769816 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:14.769802 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5886fa4-5da2-4f2b-9cbc-5b8927b903c6" containerName="test-curl-7326-140510219594800" Jul 13 20:59:14.773345 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:14.773322 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225509232" Jul 13 20:59:14.775147 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:14.775125 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7326-140510225509232"] Jul 13 20:59:14.775597 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:14.775577 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-kxvn6\"" Jul 13 20:59:14.859186 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:14.859150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkbw5\" (UniqueName: \"kubernetes.io/projected/36e30da2-52e5-4145-acf4-b215b8b0d88d-kube-api-access-xkbw5\") pod \"test-curl-7326-140510225509232\" (UID: \"36e30da2-52e5-4145-acf4-b215b8b0d88d\") " pod="openshift-ingress/test-curl-7326-140510225509232" Jul 13 20:59:14.960260 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:14.960226 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkbw5\" (UniqueName: \"kubernetes.io/projected/36e30da2-52e5-4145-acf4-b215b8b0d88d-kube-api-access-xkbw5\") pod \"test-curl-7326-140510225509232\" (UID: \"36e30da2-52e5-4145-acf4-b215b8b0d88d\") " pod="openshift-ingress/test-curl-7326-140510225509232" Jul 13 20:59:14.967817 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:14.967785 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkbw5\" (UniqueName: \"kubernetes.io/projected/36e30da2-52e5-4145-acf4-b215b8b0d88d-kube-api-access-xkbw5\") pod \"test-curl-7326-140510225509232\" (UID: \"36e30da2-52e5-4145-acf4-b215b8b0d88d\") " pod="openshift-ingress/test-curl-7326-140510225509232" Jul 13 20:59:15.085245 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:15.085208 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225509232" Jul 13 20:59:15.225685 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:15.225660 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7326-140510225509232"] Jul 13 20:59:15.226793 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:59:15.226772 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36e30da2_52e5_4145_acf4_b215b8b0d88d.slice/crio-f8ef259aa6caaf5aa2b573b3ba71092eeefb8cd8aae1bb0491a955e29394a278 WatchSource:0}: Error finding container f8ef259aa6caaf5aa2b573b3ba71092eeefb8cd8aae1bb0491a955e29394a278: Status 404 returned error can't find the container with id f8ef259aa6caaf5aa2b573b3ba71092eeefb8cd8aae1bb0491a955e29394a278 Jul 13 20:59:15.851254 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:15.851219 2568 generic.go:356] "Generic (PLEG): container finished" podID="36e30da2-52e5-4145-acf4-b215b8b0d88d" containerID="e9a05da18dbe1754cf550622b4b163f2ed37286df10b369ebcd25fc83ce78c35" exitCode=0 Jul 13 20:59:15.851702 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:15.851306 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7326-140510225509232" event={"ID":"36e30da2-52e5-4145-acf4-b215b8b0d88d","Type":"ContainerDied","Data":"e9a05da18dbe1754cf550622b4b163f2ed37286df10b369ebcd25fc83ce78c35"} Jul 13 20:59:15.851702 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:15.851362 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7326-140510225509232" event={"ID":"36e30da2-52e5-4145-acf4-b215b8b0d88d","Type":"ContainerStarted","Data":"f8ef259aa6caaf5aa2b573b3ba71092eeefb8cd8aae1bb0491a955e29394a278"} Jul 13 20:59:17.004700 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:17.004679 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225509232" Jul 13 20:59:17.023997 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:17.023960 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7326-140510225509232_36e30da2-52e5-4145-acf4-b215b8b0d88d/test-curl-7326-140510225509232/0.log" Jul 13 20:59:17.048772 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:17.048727 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7326-140510225509232"] Jul 13 20:59:17.052466 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:17.052435 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7326-140510225509232"] Jul 13 20:59:17.083115 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:17.083081 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkbw5\" (UniqueName: \"kubernetes.io/projected/36e30da2-52e5-4145-acf4-b215b8b0d88d-kube-api-access-xkbw5\") pod \"36e30da2-52e5-4145-acf4-b215b8b0d88d\" (UID: \"36e30da2-52e5-4145-acf4-b215b8b0d88d\") " Jul 13 20:59:17.085168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:17.085147 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36e30da2-52e5-4145-acf4-b215b8b0d88d-kube-api-access-xkbw5" (OuterVolumeSpecName: "kube-api-access-xkbw5") pod "36e30da2-52e5-4145-acf4-b215b8b0d88d" (UID: "36e30da2-52e5-4145-acf4-b215b8b0d88d"). InnerVolumeSpecName "kube-api-access-xkbw5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:59:17.184527 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:17.184447 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkbw5\" (UniqueName: \"kubernetes.io/projected/36e30da2-52e5-4145-acf4-b215b8b0d88d-kube-api-access-xkbw5\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:59:17.403677 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:17.403641 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36e30da2-52e5-4145-acf4-b215b8b0d88d" path="/var/lib/kubelet/pods/36e30da2-52e5-4145-acf4-b215b8b0d88d/volumes" Jul 13 20:59:17.863177 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:17.863152 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225509232" Jul 13 20:59:17.863327 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:17.863209 2568 scope.go:117] "RemoveContainer" containerID="e9a05da18dbe1754cf550622b4b163f2ed37286df10b369ebcd25fc83ce78c35" Jul 13 20:59:19.762925 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:19.762889 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7326-140510225510240"] Jul 13 20:59:19.763368 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:19.763354 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e30da2-52e5-4145-acf4-b215b8b0d88d" containerName="test-curl-7326-140510225509232" Jul 13 20:59:19.763445 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:19.763369 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e30da2-52e5-4145-acf4-b215b8b0d88d" containerName="test-curl-7326-140510225509232" Jul 13 20:59:19.763487 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:19.763458 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e30da2-52e5-4145-acf4-b215b8b0d88d" containerName="test-curl-7326-140510225509232" Jul 13 20:59:19.768193 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:19.768176 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:19.770986 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:19.770964 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-kxvn6\"" Jul 13 20:59:19.772273 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:19.772247 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7326-140510225510240"] Jul 13 20:59:19.914219 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:19.914183 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj4jn\" (UniqueName: \"kubernetes.io/projected/ff5b37db-f15a-48e6-abf1-9ead77641e8d-kube-api-access-xj4jn\") pod \"test-curl-7326-140510225510240\" (UID: \"ff5b37db-f15a-48e6-abf1-9ead77641e8d\") " pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:20.015690 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:20.015580 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj4jn\" (UniqueName: \"kubernetes.io/projected/ff5b37db-f15a-48e6-abf1-9ead77641e8d-kube-api-access-xj4jn\") pod \"test-curl-7326-140510225510240\" (UID: \"ff5b37db-f15a-48e6-abf1-9ead77641e8d\") " pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:20.023407 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:20.023380 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj4jn\" (UniqueName: \"kubernetes.io/projected/ff5b37db-f15a-48e6-abf1-9ead77641e8d-kube-api-access-xj4jn\") pod \"test-curl-7326-140510225510240\" (UID: \"ff5b37db-f15a-48e6-abf1-9ead77641e8d\") " pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:20.079503 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:20.079468 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:20.225213 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:20.225184 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7326-140510225510240"] Jul 13 20:59:20.227802 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:59:20.227771 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff5b37db_f15a_48e6_abf1_9ead77641e8d.slice/crio-7324658b5b01dffa0c768d5843aebab1c231ac99833f2fd1a93b192c6fd49f79 WatchSource:0}: Error finding container 7324658b5b01dffa0c768d5843aebab1c231ac99833f2fd1a93b192c6fd49f79: Status 404 returned error can't find the container with id 7324658b5b01dffa0c768d5843aebab1c231ac99833f2fd1a93b192c6fd49f79 Jul 13 20:59:20.881919 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:20.881827 2568 generic.go:356] "Generic (PLEG): container finished" podID="ff5b37db-f15a-48e6-abf1-9ead77641e8d" containerID="f74f77f7c4e8d4120bf9d186d9ad962fac95be2581235e93deced495f9d09102" exitCode=0 Jul 13 20:59:20.882289 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:20.881915 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7326-140510225510240" event={"ID":"ff5b37db-f15a-48e6-abf1-9ead77641e8d","Type":"ContainerDied","Data":"f74f77f7c4e8d4120bf9d186d9ad962fac95be2581235e93deced495f9d09102"} Jul 13 20:59:20.882289 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:20.881955 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7326-140510225510240" event={"ID":"ff5b37db-f15a-48e6-abf1-9ead77641e8d","Type":"ContainerStarted","Data":"7324658b5b01dffa0c768d5843aebab1c231ac99833f2fd1a93b192c6fd49f79"} Jul 13 20:59:22.032540 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.032516 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:22.051740 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.051698 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7326-140510225510240_ff5b37db-f15a-48e6-abf1-9ead77641e8d/test-curl-7326-140510225510240/0.log" Jul 13 20:59:22.074174 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.074133 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7326-140510225510240"] Jul 13 20:59:22.077717 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.077694 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7326-140510225510240"] Jul 13 20:59:22.140051 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.139978 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xj4jn\" (UniqueName: \"kubernetes.io/projected/ff5b37db-f15a-48e6-abf1-9ead77641e8d-kube-api-access-xj4jn\") pod \"ff5b37db-f15a-48e6-abf1-9ead77641e8d\" (UID: \"ff5b37db-f15a-48e6-abf1-9ead77641e8d\") " Jul 13 20:59:22.142019 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.141987 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff5b37db-f15a-48e6-abf1-9ead77641e8d-kube-api-access-xj4jn" (OuterVolumeSpecName: "kube-api-access-xj4jn") pod "ff5b37db-f15a-48e6-abf1-9ead77641e8d" (UID: "ff5b37db-f15a-48e6-abf1-9ead77641e8d"). InnerVolumeSpecName "kube-api-access-xj4jn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:59:22.196870 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.196841 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7326-140510225510240"] Jul 13 20:59:22.197295 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.197282 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff5b37db-f15a-48e6-abf1-9ead77641e8d" containerName="test-curl-7326-140510225510240" Jul 13 20:59:22.197338 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.197296 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff5b37db-f15a-48e6-abf1-9ead77641e8d" containerName="test-curl-7326-140510225510240" Jul 13 20:59:22.197389 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.197380 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff5b37db-f15a-48e6-abf1-9ead77641e8d" containerName="test-curl-7326-140510225510240" Jul 13 20:59:22.200498 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.200484 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:22.206236 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.206209 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7326-140510225510240"] Jul 13 20:59:22.241597 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.241559 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xj4jn\" (UniqueName: \"kubernetes.io/projected/ff5b37db-f15a-48e6-abf1-9ead77641e8d-kube-api-access-xj4jn\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:59:22.342467 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.342427 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrk78\" (UniqueName: \"kubernetes.io/projected/619b8478-fc6f-4516-b849-f27c5a4b00e6-kube-api-access-rrk78\") pod \"test-curl-7326-140510225510240\" (UID: \"619b8478-fc6f-4516-b849-f27c5a4b00e6\") " pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:22.443303 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.443210 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrk78\" (UniqueName: \"kubernetes.io/projected/619b8478-fc6f-4516-b849-f27c5a4b00e6-kube-api-access-rrk78\") pod \"test-curl-7326-140510225510240\" (UID: \"619b8478-fc6f-4516-b849-f27c5a4b00e6\") " pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:22.451168 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.451134 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrk78\" (UniqueName: \"kubernetes.io/projected/619b8478-fc6f-4516-b849-f27c5a4b00e6-kube-api-access-rrk78\") pod \"test-curl-7326-140510225510240\" (UID: \"619b8478-fc6f-4516-b849-f27c5a4b00e6\") " pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:22.510181 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.510147 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:22.646656 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.646628 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7326-140510225510240"] Jul 13 20:59:22.647506 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:59:22.647481 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod619b8478_fc6f_4516_b849_f27c5a4b00e6.slice/crio-1f597b69af875e41ad42a2dec89c609af30db3089dc2f6d2b8860cdc9313cadd WatchSource:0}: Error finding container 1f597b69af875e41ad42a2dec89c609af30db3089dc2f6d2b8860cdc9313cadd: Status 404 returned error can't find the container with id 1f597b69af875e41ad42a2dec89c609af30db3089dc2f6d2b8860cdc9313cadd Jul 13 20:59:22.896067 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.896024 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7326-140510225510240" event={"ID":"619b8478-fc6f-4516-b849-f27c5a4b00e6","Type":"ContainerStarted","Data":"1f597b69af875e41ad42a2dec89c609af30db3089dc2f6d2b8860cdc9313cadd"} Jul 13 20:59:22.897267 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.897245 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7324658b5b01dffa0c768d5843aebab1c231ac99833f2fd1a93b192c6fd49f79" Jul 13 20:59:22.897267 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.897265 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:22.908488 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:22.908448 2568 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7326-140510225510240" oldPodUID="ff5b37db-f15a-48e6-abf1-9ead77641e8d" podUID="619b8478-fc6f-4516-b849-f27c5a4b00e6" Jul 13 20:59:23.407119 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:23.407087 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff5b37db-f15a-48e6-abf1-9ead77641e8d" path="/var/lib/kubelet/pods/ff5b37db-f15a-48e6-abf1-9ead77641e8d/volumes" Jul 13 20:59:23.905921 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:23.905888 2568 generic.go:356] "Generic (PLEG): container finished" podID="619b8478-fc6f-4516-b849-f27c5a4b00e6" containerID="d0b60d0ddae9d15f0a6f823714106ffc5735c0f1a5f07c83d6d54bf49533f7c1" exitCode=0 Jul 13 20:59:23.906375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:23.905974 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7326-140510225510240" event={"ID":"619b8478-fc6f-4516-b849-f27c5a4b00e6","Type":"ContainerDied","Data":"d0b60d0ddae9d15f0a6f823714106ffc5735c0f1a5f07c83d6d54bf49533f7c1"} Jul 13 20:59:25.058637 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.058612 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:25.078298 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.078265 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7326-140510225510240_619b8478-fc6f-4516-b849-f27c5a4b00e6/test-curl-7326-140510225510240/0.log" Jul 13 20:59:25.103014 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.102979 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7326-140510225510240"] Jul 13 20:59:25.106530 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.106505 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7326-140510225510240"] Jul 13 20:59:25.173717 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.173620 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrk78\" (UniqueName: \"kubernetes.io/projected/619b8478-fc6f-4516-b849-f27c5a4b00e6-kube-api-access-rrk78\") pod \"619b8478-fc6f-4516-b849-f27c5a4b00e6\" (UID: \"619b8478-fc6f-4516-b849-f27c5a4b00e6\") " Jul 13 20:59:25.175713 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.175691 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/619b8478-fc6f-4516-b849-f27c5a4b00e6-kube-api-access-rrk78" (OuterVolumeSpecName: "kube-api-access-rrk78") pod "619b8478-fc6f-4516-b849-f27c5a4b00e6" (UID: "619b8478-fc6f-4516-b849-f27c5a4b00e6"). InnerVolumeSpecName "kube-api-access-rrk78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:59:25.274779 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.274742 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrk78\" (UniqueName: \"kubernetes.io/projected/619b8478-fc6f-4516-b849-f27c5a4b00e6-kube-api-access-rrk78\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:59:25.295030 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.294995 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945"] Jul 13 20:59:25.295549 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.295531 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619b8478-fc6f-4516-b849-f27c5a4b00e6" containerName="test-curl-7326-140510225510240" Jul 13 20:59:25.295671 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.295551 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="619b8478-fc6f-4516-b849-f27c5a4b00e6" containerName="test-curl-7326-140510225510240" Jul 13 20:59:25.295730 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.295694 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="619b8478-fc6f-4516-b849-f27c5a4b00e6" containerName="test-curl-7326-140510225510240" Jul 13 20:59:25.299020 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.298999 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945" Jul 13 20:59:25.301344 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.301327 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-84mvh\"" Jul 13 20:59:25.305854 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.305828 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945"] Jul 13 20:59:25.404151 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.404105 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="619b8478-fc6f-4516-b849-f27c5a4b00e6" path="/var/lib/kubelet/pods/619b8478-fc6f-4516-b849-f27c5a4b00e6/volumes" Jul 13 20:59:25.477608 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.477512 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0502c49-895c-4a2a-a6c5-30fc898a5f0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-264ac7-lg945\" (UID: \"e0502c49-895c-4a2a-a6c5-30fc898a5f0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945" Jul 13 20:59:25.578888 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.578853 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0502c49-895c-4a2a-a6c5-30fc898a5f0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-264ac7-lg945\" (UID: \"e0502c49-895c-4a2a-a6c5-30fc898a5f0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945" Jul 13 20:59:25.579507 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.579488 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0502c49-895c-4a2a-a6c5-30fc898a5f0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-264ac7-lg945\" (UID: \"e0502c49-895c-4a2a-a6c5-30fc898a5f0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945" Jul 13 20:59:25.612738 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.612701 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945" Jul 13 20:59:25.765956 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.765930 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945"] Jul 13 20:59:25.767552 ip-10-0-135-149 kubenswrapper[2568]: W0713 20:59:25.767523 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0502c49_895c_4a2a_a6c5_30fc898a5f0b.slice/crio-94f1992d92d5638d98d59b3ce38c72c62f20b18c037bba60cc916832b464eca9 WatchSource:0}: Error finding container 94f1992d92d5638d98d59b3ce38c72c62f20b18c037bba60cc916832b464eca9: Status 404 returned error can't find the container with id 94f1992d92d5638d98d59b3ce38c72c62f20b18c037bba60cc916832b464eca9 Jul 13 20:59:25.918225 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.918193 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7326-140510225510240" Jul 13 20:59:25.918456 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.918253 2568 scope.go:117] "RemoveContainer" containerID="d0b60d0ddae9d15f0a6f823714106ffc5735c0f1a5f07c83d6d54bf49533f7c1" Jul 13 20:59:25.919682 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:25.919660 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945" event={"ID":"e0502c49-895c-4a2a-a6c5-30fc898a5f0b","Type":"ContainerStarted","Data":"94f1992d92d5638d98d59b3ce38c72c62f20b18c037bba60cc916832b464eca9"} Jul 13 20:59:26.927459 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:26.927342 2568 generic.go:356] "Generic (PLEG): container finished" podID="e0502c49-895c-4a2a-a6c5-30fc898a5f0b" containerID="962971b110243a6cb0446e52f4ccff6180015f5659a4778bb707fcc51899d1ad" exitCode=0 Jul 13 20:59:26.927459 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:26.927443 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945" event={"ID":"e0502c49-895c-4a2a-a6c5-30fc898a5f0b","Type":"ContainerDied","Data":"962971b110243a6cb0446e52f4ccff6180015f5659a4778bb707fcc51899d1ad"} Jul 13 20:59:28.076794 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:28.076765 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945" Jul 13 20:59:28.206273 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:28.206183 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0502c49-895c-4a2a-a6c5-30fc898a5f0b-maas-api-service-ca\") pod \"e0502c49-895c-4a2a-a6c5-30fc898a5f0b\" (UID: \"e0502c49-895c-4a2a-a6c5-30fc898a5f0b\") " Jul 13 20:59:28.206629 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:28.206601 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0502c49-895c-4a2a-a6c5-30fc898a5f0b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e0502c49-895c-4a2a-a6c5-30fc898a5f0b" (UID: "e0502c49-895c-4a2a-a6c5-30fc898a5f0b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:59:28.307889 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:28.307854 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0502c49-895c-4a2a-a6c5-30fc898a5f0b-maas-api-service-ca\") on node \"ip-10-0-135-149.ec2.internal\" DevicePath \"\"" Jul 13 20:59:28.944973 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:28.944942 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945" Jul 13 20:59:28.944973 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:28.944951 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-264ac7-lg945" event={"ID":"e0502c49-895c-4a2a-a6c5-30fc898a5f0b","Type":"ContainerDied","Data":"94f1992d92d5638d98d59b3ce38c72c62f20b18c037bba60cc916832b464eca9"} Jul 13 20:59:28.945180 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:28.945000 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94f1992d92d5638d98d59b3ce38c72c62f20b18c037bba60cc916832b464eca9" Jul 13 20:59:29.562057 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:29.562021 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 20:59:29.572375 ip-10-0-135-149 kubenswrapper[2568]: I0713 20:59:29.570776 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nblwk_94fa8c97-cdf1-4b58-9fdb-5a4de75009ae/ovn-acl-logging/0.log" Jul 13 21:00:00.137377 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.137352 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn"] Jul 13 21:00:00.137937 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.137820 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0502c49-895c-4a2a-a6c5-30fc898a5f0b" containerName="revoke-keys" Jul 13 21:00:00.137937 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.137832 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0502c49-895c-4a2a-a6c5-30fc898a5f0b" containerName="revoke-keys" Jul 13 21:00:00.137937 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.137919 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0502c49-895c-4a2a-a6c5-30fc898a5f0b" containerName="revoke-keys" Jul 13 21:00:00.140304 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.140288 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll"] Jul 13 21:00:00.140424 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.140398 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn" Jul 13 21:00:00.142627 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.142605 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-84mvh\"" Jul 13 21:00:00.143351 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.143332 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll" Jul 13 21:00:00.146226 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.146193 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn"] Jul 13 21:00:00.149535 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.149515 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll"] Jul 13 21:00:00.154348 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.154328 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn" Jul 13 21:00:00.161497 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.161479 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll" Jul 13 21:00:00.322696 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.322666 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn"] Jul 13 21:00:00.325371 ip-10-0-135-149 kubenswrapper[2568]: W0713 21:00:00.325334 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod274e0d9b_2823_42a8_90c1_b66e7188d1c6.slice/crio-4a891fea6db16e88a9a8bf3171c335adba183bcd54400e1b46d214b8e63fca46 WatchSource:0}: Error finding container 4a891fea6db16e88a9a8bf3171c335adba183bcd54400e1b46d214b8e63fca46: Status 404 returned error can't find the container with id 4a891fea6db16e88a9a8bf3171c335adba183bcd54400e1b46d214b8e63fca46 Jul 13 21:00:00.347797 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:00.347773 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll"] Jul 13 21:00:00.349703 ip-10-0-135-149 kubenswrapper[2568]: W0713 21:00:00.349677 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a95ded_4936_4bc6_98f8_a297465da5bc.slice/crio-63a3eb8113938acf5ace657462896efa89387593c4a669cd443273d07f091416 WatchSource:0}: Error finding container 63a3eb8113938acf5ace657462896efa89387593c4a669cd443273d07f091416: Status 404 returned error can't find the container with id 63a3eb8113938acf5ace657462896efa89387593c4a669cd443273d07f091416 Jul 13 21:00:01.133907 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:01.133866 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn" event={"ID":"274e0d9b-2823-42a8-90c1-b66e7188d1c6","Type":"ContainerStarted","Data":"40db380da194bc1d517fa2e9693c6ee1d5bf9bb1365fd0f3c0c842ea64149f38"} Jul 13 21:00:01.134117 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:01.133931 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn" event={"ID":"274e0d9b-2823-42a8-90c1-b66e7188d1c6","Type":"ContainerStarted","Data":"4a891fea6db16e88a9a8bf3171c335adba183bcd54400e1b46d214b8e63fca46"} Jul 13 21:00:01.135532 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:01.135504 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll" event={"ID":"78a95ded-4936-4bc6-98f8-a297465da5bc","Type":"ContainerStarted","Data":"0eb4ba0fd985728ddb47bd44ce7b899945b61a0ab6ee6416f6e8243b6438aeac"} Jul 13 21:00:01.135671 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:01.135568 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll" event={"ID":"78a95ded-4936-4bc6-98f8-a297465da5bc","Type":"ContainerStarted","Data":"63a3eb8113938acf5ace657462896efa89387593c4a669cd443273d07f091416"} Jul 13 21:00:01.147144 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:01.147087 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn" podStartSLOduration=1.147068135 podStartE2EDuration="1.147068135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:00:01.146652326 +0000 UTC m=+2732.292919488" watchObservedRunningTime="2026-07-13 21:00:01.147068135 +0000 UTC m=+2732.293335297" Jul 13 21:00:01.161335 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:01.161288 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll" podStartSLOduration=1.161272468 podStartE2EDuration="1.161272468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:00:01.159504767 +0000 UTC m=+2732.305771929" watchObservedRunningTime="2026-07-13 21:00:01.161272468 +0000 UTC m=+2732.307539631" Jul 13 21:00:01.858497 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:01.858462 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn"] Jul 13 21:00:02.144542 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:02.144447 2568 generic.go:356] "Generic (PLEG): container finished" podID="78a95ded-4936-4bc6-98f8-a297465da5bc" containerID="0eb4ba0fd985728ddb47bd44ce7b899945b61a0ab6ee6416f6e8243b6438aeac" exitCode=0 Jul 13 21:00:02.144696 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:02.144540 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll" event={"ID":"78a95ded-4936-4bc6-98f8-a297465da5bc","Type":"ContainerDied","Data":"0eb4ba0fd985728ddb47bd44ce7b899945b61a0ab6ee6416f6e8243b6438aeac"} Jul 13 21:00:02.146186 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:02.146164 2568 generic.go:356] "Generic (PLEG): container finished" podID="274e0d9b-2823-42a8-90c1-b66e7188d1c6" containerID="40db380da194bc1d517fa2e9693c6ee1d5bf9bb1365fd0f3c0c842ea64149f38" exitCode=0 Jul 13 21:00:02.146319 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:02.146192 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn" event={"ID":"274e0d9b-2823-42a8-90c1-b66e7188d1c6","Type":"ContainerDied","Data":"40db380da194bc1d517fa2e9693c6ee1d5bf9bb1365fd0f3c0c842ea64149f38"} Jul 13 21:00:03.330155 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:03.330128 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn" Jul 13 21:00:03.333214 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:03.333190 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll" Jul 13 21:00:04.158544 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:04.158515 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn" Jul 13 21:00:04.158723 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:04.158554 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn" event={"ID":"274e0d9b-2823-42a8-90c1-b66e7188d1c6","Type":"ContainerDied","Data":"4a891fea6db16e88a9a8bf3171c335adba183bcd54400e1b46d214b8e63fca46"} Jul 13 21:00:04.158723 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:04.158614 2568 scope.go:117] "RemoveContainer" containerID="40db380da194bc1d517fa2e9693c6ee1d5bf9bb1365fd0f3c0c842ea64149f38" Jul 13 21:00:04.159957 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:04.159935 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll" event={"ID":"78a95ded-4936-4bc6-98f8-a297465da5bc","Type":"ContainerDied","Data":"63a3eb8113938acf5ace657462896efa89387593c4a669cd443273d07f091416"} Jul 13 21:00:04.160010 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:04.159981 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63a3eb8113938acf5ace657462896efa89387593c4a669cd443273d07f091416" Jul 13 21:00:04.160010 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:04.159947 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732940-8m4ll" Jul 13 21:00:04.178867 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:04.178840 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn"] Jul 13 21:00:04.181202 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:04.181182 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-11843d-29732940-l62qn"] Jul 13 21:00:05.403543 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:05.403501 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="274e0d9b-2823-42a8-90c1-b66e7188d1c6" path="/var/lib/kubelet/pods/274e0d9b-2823-42a8-90c1-b66e7188d1c6/volumes" Jul 13 21:00:28.062729 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:28.062693 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-pfcq2_01e36626-7a98-451d-ade1-60059a0e6d7f/manager/0.log" Jul 13 21:00:28.188712 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:28.188675 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-z6ndr_0ddbc65d-796f-4dbc-a870-b079c377a278/manager/0.log" Jul 13 21:00:28.321345 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:28.321256 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d466f49f4-wgb26_28c12228-afd5-4a1d-8915-7b89239b332d/manager/0.log" Jul 13 21:00:28.943245 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:28.943211 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b56dd55f-p7gf5_85f52de8-87a0-42fc-a855-dbd5d0282791/copy-manifests/0.log" Jul 13 21:00:28.950458 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:28.950405 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b56dd55f-p7gf5_85f52de8-87a0-42fc-a855-dbd5d0282791/manager/0.log" Jul 13 21:00:29.308536 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:29.308475 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5889597b4-c6hps_916a932f-8302-43c3-a725-c00d0a906d19/maas-api/0.log" Jul 13 21:00:29.647437 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:29.647337 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29732940-8m4ll_78a95ded-4936-4bc6-98f8-a297465da5bc/cleanup/0.log" Jul 13 21:00:29.759317 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:29.759284 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-9ebcdac6-b2msb_e8bc682d-f1e1-4c44-8e90-634b07dadde5/revoke-keys/0.log" Jul 13 21:00:29.870647 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:29.870616 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-b435b208-x2wbq_c0501640-c850-43db-b25c-4f6e0c3fb447/revoke-keys/1.log" Jul 13 21:00:29.989984 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:29.989904 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-ad7818-9lh48_e7477a21-dc26-4e02-a453-b8e1802f8d39/revoke-keys/0.log" Jul 13 21:00:30.331586 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:30.331547 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-abc63a-g8b79_7818c224-90ef-4930-8adb-0be7e81f4de0/revoke-keys/0.log" Jul 13 21:00:30.454559 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:30.454522 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-a35e15-6znhd_cc87d7e3-ea44-4bbf-a537-3ce888f9af06/revoke-keys/0.log" Jul 13 21:00:30.572386 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:30.572352 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-19b1db0a-z2ccs_33dd272f-7f51-42fd-9848-97920b412bc7/revoke-keys/0.log" Jul 13 21:00:30.684299 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:30.684212 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-d387c72d-h69ln_29b99cf1-f8db-4f0a-9308-1b7bda8492e7/revoke-keys/0.log" Jul 13 21:00:30.798906 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:30.798874 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-264ac7-lg945_e0502c49-895c-4a2a-a6c5-30fc898a5f0b/revoke-keys/0.log" Jul 13 21:00:31.038523 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:31.038430 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-j629t_b6897745-694f-412d-9faa-c71e0d36dfbf/postgres/0.log" Jul 13 21:00:32.815999 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:32.815955 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6rxrv_8465cb61-31ce-466a-acdd-6141c94bd720/registry-server/0.log" Jul 13 21:00:32.951340 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:32.951183 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tl2kv_57c8b1a6-aae9-40e7-94b1-c079ed126500/manager/0.log" Jul 13 21:00:34.026898 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:34.026863 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8g8t_1071df1c-06ea-4b49-9ca9-f1f504a09d75/istio-proxy/1.log" Jul 13 21:00:34.151830 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:34.151795 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-nt7xm_301e4b2f-38c4-44df-be14-3ffa9238e143/payload-pre-processing/0.log" Jul 13 21:00:34.273692 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:34.273642 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-2zm5j_f508a9e7-526d-449a-9298-562e1f74b2fa/payload-processing/0.log" Jul 13 21:00:34.754499 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:34.754468 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-9cfqk_3db964e8-9153-4bbb-974d-fbea7ecacb04/main/0.log" Jul 13 21:00:34.989203 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:34.989174 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-ckqxd_08a87a37-0b60-4e14-b135-1c467565e46a/main/0.log" Jul 13 21:00:35.106405 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:35.106375 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db48pdz_346edc07-31a5-417c-9135-c9e8b559c559/main/0.log" Jul 13 21:00:42.181614 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:42.181577 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4hjb2_09d72f4e-4a0a-45c3-be16-f3c675544067/global-pull-secret-syncer/0.log" Jul 13 21:00:42.384226 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:42.384195 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-plm24_d49f5e7f-dc27-40e2-a16b-a738f7a23ce3/konnectivity-agent/0.log" Jul 13 21:00:42.432478 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:42.432361 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-149.ec2.internal_66cb28dc9c7115d14af06ad19fd0f8b8/haproxy/0.log" Jul 13 21:00:47.486647 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:47.486605 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6rxrv_8465cb61-31ce-466a-acdd-6141c94bd720/registry-server/0.log" Jul 13 21:00:47.718823 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:47.718786 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tl2kv_57c8b1a6-aae9-40e7-94b1-c079ed126500/manager/0.log" Jul 13 21:00:49.323782 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.323750 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81974efb-ad6d-4280-a040-5aa6e357721d/alertmanager/0.log" Jul 13 21:00:49.343191 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.343164 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81974efb-ad6d-4280-a040-5aa6e357721d/config-reloader/0.log" Jul 13 21:00:49.364505 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.364452 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81974efb-ad6d-4280-a040-5aa6e357721d/kube-rbac-proxy-web/0.log" Jul 13 21:00:49.385274 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.385245 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81974efb-ad6d-4280-a040-5aa6e357721d/kube-rbac-proxy/0.log" Jul 13 21:00:49.410936 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.410907 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81974efb-ad6d-4280-a040-5aa6e357721d/kube-rbac-proxy-metric/0.log" Jul 13 21:00:49.437388 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.437355 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81974efb-ad6d-4280-a040-5aa6e357721d/prom-label-proxy/0.log" Jul 13 21:00:49.457639 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.457604 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81974efb-ad6d-4280-a040-5aa6e357721d/init-config-reloader/0.log" Jul 13 21:00:49.536648 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.536620 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-txkzk_785caab8-d70e-4b51-b02e-e3b54830e32d/kube-state-metrics/0.log" Jul 13 21:00:49.551355 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.551320 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-txkzk_785caab8-d70e-4b51-b02e-e3b54830e32d/kube-rbac-proxy-main/0.log" Jul 13 21:00:49.572000 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.571967 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-txkzk_785caab8-d70e-4b51-b02e-e3b54830e32d/kube-rbac-proxy-self/0.log" Jul 13 21:00:49.602165 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.602069 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c8d7d54c4-hxbvs_8f5e9872-59e3-4721-888c-999267360e49/metrics-server/0.log" Jul 13 21:00:49.625020 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.624984 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-qj4jz_852f60a6-e917-4e05-b249-b48a640f78e8/monitoring-plugin/0.log" Jul 13 21:00:49.738807 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.738772 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dqw5l_97e64137-f19c-4c25-9667-b62d069196aa/node-exporter/0.log" Jul 13 21:00:49.759025 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.759005 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dqw5l_97e64137-f19c-4c25-9667-b62d069196aa/kube-rbac-proxy/0.log" Jul 13 21:00:49.780154 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.780125 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dqw5l_97e64137-f19c-4c25-9667-b62d069196aa/init-textfile/0.log" Jul 13 21:00:49.900977 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.900891 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-p7kkn_02448df2-52e4-4456-8d7f-d5b22bbaaa90/kube-rbac-proxy-main/0.log" Jul 13 21:00:49.928404 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.928353 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-p7kkn_02448df2-52e4-4456-8d7f-d5b22bbaaa90/kube-rbac-proxy-self/0.log" Jul 13 21:00:49.959825 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:49.959794 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-p7kkn_02448df2-52e4-4456-8d7f-d5b22bbaaa90/openshift-state-metrics/0.log" Jul 13 21:00:50.004915 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.004825 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1/prometheus/0.log" Jul 13 21:00:50.022915 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.022888 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1/config-reloader/0.log" Jul 13 21:00:50.041929 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.041888 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1/thanos-sidecar/0.log" Jul 13 21:00:50.060696 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.060672 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1/kube-rbac-proxy-web/0.log" Jul 13 21:00:50.080546 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.080518 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1/kube-rbac-proxy/0.log" Jul 13 21:00:50.098751 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.098722 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1/kube-rbac-proxy-thanos/0.log" Jul 13 21:00:50.122185 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.122160 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1fa7f0f6-7f77-4086-ae6d-5d7af9bdf1b1/init-config-reloader/0.log" Jul 13 21:00:50.156486 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.156378 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-gj22q_8e2b8767-9ad0-42ec-bcb2-cfb2340fe190/prometheus-operator/0.log" Jul 13 21:00:50.171317 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.171275 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-gj22q_8e2b8767-9ad0-42ec-bcb2-cfb2340fe190/kube-rbac-proxy/0.log" Jul 13 21:00:50.195439 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.195392 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-68lbd_56f1bc0c-34ee-40c3-b9d7-e45ccf74413d/prometheus-operator-admission-webhook/0.log" Jul 13 21:00:50.229443 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.229394 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b547fd5f-mmcvv_0c2f3ae3-90de-403b-b7fd-050f4ec77f47/telemeter-client/0.log" Jul 13 21:00:50.254763 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.254726 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b547fd5f-mmcvv_0c2f3ae3-90de-403b-b7fd-050f4ec77f47/reload/0.log" Jul 13 21:00:50.285636 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.285611 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b547fd5f-mmcvv_0c2f3ae3-90de-403b-b7fd-050f4ec77f47/kube-rbac-proxy/0.log" Jul 13 21:00:50.331067 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.331026 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-867d97cf76-ljlbj_35d75eb7-4b6f-4270-a7cc-d99f9291555e/thanos-query/0.log" Jul 13 21:00:50.351792 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.351760 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-867d97cf76-ljlbj_35d75eb7-4b6f-4270-a7cc-d99f9291555e/kube-rbac-proxy-web/0.log" Jul 13 21:00:50.372894 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.372862 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-867d97cf76-ljlbj_35d75eb7-4b6f-4270-a7cc-d99f9291555e/kube-rbac-proxy/0.log" Jul 13 21:00:50.393237 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.393211 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-867d97cf76-ljlbj_35d75eb7-4b6f-4270-a7cc-d99f9291555e/prom-label-proxy/0.log" Jul 13 21:00:50.412915 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.412829 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-867d97cf76-ljlbj_35d75eb7-4b6f-4270-a7cc-d99f9291555e/kube-rbac-proxy-rules/0.log" Jul 13 21:00:50.432431 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:50.432386 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-867d97cf76-ljlbj_35d75eb7-4b6f-4270-a7cc-d99f9291555e/kube-rbac-proxy-metrics/0.log" Jul 13 21:00:52.770033 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:52.770006 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b5468d855-77cv6_5cc9cddc-8608-4a81-8669-6538645e91a1/console/0.log" Jul 13 21:00:52.810297 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:52.810262 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-q6lng_8a35a35a-0052-4fa5-9d36-bb8eb78b4c22/download-server/0.log" Jul 13 21:00:53.004242 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.004193 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c"] Jul 13 21:00:53.004900 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.004869 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="274e0d9b-2823-42a8-90c1-b66e7188d1c6" containerName="cleanup" Jul 13 21:00:53.004900 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.004891 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="274e0d9b-2823-42a8-90c1-b66e7188d1c6" containerName="cleanup" Jul 13 21:00:53.004900 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.004902 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78a95ded-4936-4bc6-98f8-a297465da5bc" containerName="cleanup" Jul 13 21:00:53.005112 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.004908 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="78a95ded-4936-4bc6-98f8-a297465da5bc" containerName="cleanup" Jul 13 21:00:53.005112 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.004998 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="78a95ded-4936-4bc6-98f8-a297465da5bc" containerName="cleanup" Jul 13 21:00:53.005112 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.005013 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="274e0d9b-2823-42a8-90c1-b66e7188d1c6" containerName="cleanup" Jul 13 21:00:53.008649 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.008630 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.010974 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.010953 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-blmgq\"/\"openshift-service-ca.crt\"" Jul 13 21:00:53.011712 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.011696 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-blmgq\"/\"kube-root-ca.crt\"" Jul 13 21:00:53.011799 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.011699 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-blmgq\"/\"default-dockercfg-8vwhj\"" Jul 13 21:00:53.020372 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.020307 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c"] Jul 13 21:00:53.071788 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.071751 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-proc\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.071788 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.071792 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsbdl\" (UniqueName: \"kubernetes.io/projected/80c76523-b91d-45c4-b0ff-a7b009d13097-kube-api-access-bsbdl\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.072067 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.071835 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-sys\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.072067 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.071920 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-podres\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.072067 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.071983 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-lib-modules\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.173153 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.173107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-proc\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.173153 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.173159 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsbdl\" (UniqueName: \"kubernetes.io/projected/80c76523-b91d-45c4-b0ff-a7b009d13097-kube-api-access-bsbdl\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.173466 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.173217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-sys\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.173466 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.173240 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-proc\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.173466 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.173259 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-podres\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.173466 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.173324 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-sys\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.173466 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.173380 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-podres\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.173466 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.173451 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-lib-modules\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.173736 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.173619 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80c76523-b91d-45c4-b0ff-a7b009d13097-lib-modules\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.180785 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.180758 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsbdl\" (UniqueName: \"kubernetes.io/projected/80c76523-b91d-45c4-b0ff-a7b009d13097-kube-api-access-bsbdl\") pod \"perf-node-gather-daemonset-gb99c\" (UID: \"80c76523-b91d-45c4-b0ff-a7b009d13097\") " pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.319896 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.319853 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:53.465931 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:53.465902 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c"] Jul 13 21:00:54.440591 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:54.440555 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-66tb9_958745c0-4511-4118-9b63-d62bf27a2317/dns/0.log" Jul 13 21:00:54.458829 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:54.458792 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-66tb9_958745c0-4511-4118-9b63-d62bf27a2317/kube-rbac-proxy/0.log" Jul 13 21:00:54.461604 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:54.461578 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" event={"ID":"80c76523-b91d-45c4-b0ff-a7b009d13097","Type":"ContainerStarted","Data":"b51eeb0aedea27d9bec58b593374da2bd9e5caa431d035109e2e36b23ac9e807"} Jul 13 21:00:54.461752 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:54.461657 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" event={"ID":"80c76523-b91d-45c4-b0ff-a7b009d13097","Type":"ContainerStarted","Data":"821028d8d3a959ac06ed8b8fc328442d19f94f4c71de1753a8a0dbf048803954"} Jul 13 21:00:54.461752 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:54.461716 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:00:54.478017 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:54.477962 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" podStartSLOduration=2.477945376 podStartE2EDuration="2.477945376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:00:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:00:54.476394244 +0000 UTC m=+2785.622661406" watchObservedRunningTime="2026-07-13 21:00:54.477945376 +0000 UTC m=+2785.624212536" Jul 13 21:00:54.593004 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:54.592975 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l8rnl_868f0604-458b-4ba7-a928-5cc701834300/dns-node-resolver/0.log" Jul 13 21:00:55.165652 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:55.165612 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fdrjt_116a96e7-2d64-44a3-8e91-6681027a23ed/node-ca/0.log" Jul 13 21:00:56.270200 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:56.270172 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8g8t_1071df1c-06ea-4b49-9ca9-f1f504a09d75/istio-proxy/0.log" Jul 13 21:00:56.330585 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:56.330553 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8g8t_1071df1c-06ea-4b49-9ca9-f1f504a09d75/istio-proxy/1.log" Jul 13 21:00:56.368327 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:56.368291 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-nt7xm_301e4b2f-38c4-44df-be14-3ffa9238e143/payload-pre-processing/0.log" Jul 13 21:00:56.406042 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:56.406013 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-2zm5j_f508a9e7-526d-449a-9298-562e1f74b2fa/payload-processing/0.log" Jul 13 21:00:57.135901 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:57.135873 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rmtq4_fc7a2600-2ef7-4a68-94dd-d129973cf5e8/serve-healthcheck-canary/0.log" Jul 13 21:00:57.615113 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:57.615085 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8w4hv_cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50/kube-rbac-proxy/0.log" Jul 13 21:00:57.635105 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:57.635076 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8w4hv_cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50/exporter/0.log" Jul 13 21:00:57.657358 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:57.657328 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8w4hv_cb3ec07b-0adb-4ced-9fd5-619a3bf5ba50/extractor/0.log" Jul 13 21:00:59.800048 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:00:59.800012 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-pfcq2_01e36626-7a98-451d-ade1-60059a0e6d7f/manager/0.log" Jul 13 21:01:00.027612 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:00.027582 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-z6ndr_0ddbc65d-796f-4dbc-a870-b079c377a278/manager/0.log" Jul 13 21:01:00.194431 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:00.194313 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d466f49f4-wgb26_28c12228-afd5-4a1d-8915-7b89239b332d/manager/0.log" Jul 13 21:01:00.462704 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:00.462620 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b56dd55f-p7gf5_85f52de8-87a0-42fc-a855-dbd5d0282791/manager/0.log" Jul 13 21:01:00.478750 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:00.478725 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-blmgq/perf-node-gather-daemonset-gb99c" Jul 13 21:01:00.480323 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:00.480304 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b56dd55f-p7gf5_85f52de8-87a0-42fc-a855-dbd5d0282791/copy-manifests/0.log" Jul 13 21:01:01.743515 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:01.743436 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7f676ff564-sdzht_81bce3da-2a6b-48c2-8af3-21d2b0a718b8/manager/0.log" Jul 13 21:01:01.771360 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:01.771335 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-vfbx5_ae7eca11-d4f9-4e8b-927b-d76de1854f0c/openshift-lws-operator/0.log" Jul 13 21:01:02.738894 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:02.738813 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8g8t_1071df1c-06ea-4b49-9ca9-f1f504a09d75/istio-proxy/0.log" Jul 13 21:01:02.796490 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:02.796455 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8g8t_1071df1c-06ea-4b49-9ca9-f1f504a09d75/istio-proxy/1.log" Jul 13 21:01:02.825217 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:02.825185 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-nt7xm_301e4b2f-38c4-44df-be14-3ffa9238e143/payload-pre-processing/0.log" Jul 13 21:01:02.853872 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:02.853846 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-2zm5j_f508a9e7-526d-449a-9298-562e1f74b2fa/payload-processing/0.log" Jul 13 21:01:04.219719 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:04.219622 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8g8t_1071df1c-06ea-4b49-9ca9-f1f504a09d75/istio-proxy/0.log" Jul 13 21:01:04.281877 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:04.281848 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8g8t_1071df1c-06ea-4b49-9ca9-f1f504a09d75/istio-proxy/1.log" Jul 13 21:01:04.311852 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:04.311815 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-nt7xm_301e4b2f-38c4-44df-be14-3ffa9238e143/payload-pre-processing/0.log" Jul 13 21:01:04.345039 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:04.345005 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-2zm5j_f508a9e7-526d-449a-9298-562e1f74b2fa/payload-processing/0.log" Jul 13 21:01:08.257310 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:08.257274 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9xn9w_d1a320fc-c21b-4a93-a1c7-d984bc40b4ee/kube-multus/0.log" Jul 13 21:01:08.292566 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:08.292537 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z8ck_d53e230c-cc46-44e1-a8f3-6cc9766fbf55/kube-multus-additional-cni-plugins/0.log" Jul 13 21:01:08.328724 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:08.328695 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z8ck_d53e230c-cc46-44e1-a8f3-6cc9766fbf55/egress-router-binary-copy/0.log" Jul 13 21:01:08.372751 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:08.372723 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z8ck_d53e230c-cc46-44e1-a8f3-6cc9766fbf55/cni-plugins/0.log" Jul 13 21:01:08.404643 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:08.404617 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z8ck_d53e230c-cc46-44e1-a8f3-6cc9766fbf55/bond-cni-plugin/0.log" Jul 13 21:01:08.440664 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:08.440632 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z8ck_d53e230c-cc46-44e1-a8f3-6cc9766fbf55/routeoverride-cni/0.log" Jul 13 21:01:08.479740 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:08.479708 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z8ck_d53e230c-cc46-44e1-a8f3-6cc9766fbf55/whereabouts-cni-bincopy/0.log" Jul 13 21:01:08.515649 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:08.515565 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z8ck_d53e230c-cc46-44e1-a8f3-6cc9766fbf55/whereabouts-cni/0.log" Jul 13 21:01:09.293183 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:09.293151 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jtdht_eaff4f62-5b33-40cf-ab33-ea75668f2ebb/network-metrics-daemon/0.log" Jul 13 21:01:09.320995 ip-10-0-135-149 kubenswrapper[2568]: I0713 21:01:09.320933 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jtdht_eaff4f62-5b33-40cf-ab33-ea75668f2ebb/kube-rbac-proxy/0.log"