Jul 07 11:15:42.295631 ip-10-0-135-166 systemd[1]: Starting Kubernetes Kubelet... Jul 07 11:15:43.366100 ip-10-0-135-166 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:15:43.366100 ip-10-0-135-166 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 11:15:43.366100 ip-10-0-135-166 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:15:43.366100 ip-10-0-135-166 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 11:15:43.366100 ip-10-0-135-166 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:15:43.366782 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.366150 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 11:15:43.372705 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372680 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:43.372705 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372703 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:43.372705 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372709 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:43.372705 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372713 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372717 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372721 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372725 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372728 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372731 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372735 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372737 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372740 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372743 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372746 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372748 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372751 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372754 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372757 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372759 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372762 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372765 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372767 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:43.372887 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372770 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372773 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372776 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372778 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372783 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372787 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372791 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372794 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372796 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372799 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372805 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372809 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372812 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372814 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372818 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372821 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372823 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372827 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372829 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372832 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:43.373352 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372835 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372838 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372840 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372843 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372846 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372849 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372851 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372854 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372857 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372860 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372862 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372865 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372868 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372870 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372873 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372877 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372882 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372886 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372890 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:43.373912 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372893 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372896 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372899 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372902 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372904 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372907 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372910 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372913 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372916 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372918 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372921 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372924 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372926 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372929 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372932 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372936 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372939 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372941 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372944 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372947 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:43.374397 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372950 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372953 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372956 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372959 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372962 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372964 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372967 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372970 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372973 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372976 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372979 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372981 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372984 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372986 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372989 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372992 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372994 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.372997 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373772 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373780 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:43.374910 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373783 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373786 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373789 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373792 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373794 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373797 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373800 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373803 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373806 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373808 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373811 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373814 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373819 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373823 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373827 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373830 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373833 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373836 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373839 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373842 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:43.375470 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373845 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373848 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373851 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373853 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373856 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373859 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373861 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373864 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373867 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373870 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373873 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373877 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373879 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373882 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373885 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373888 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373892 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373895 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373897 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373900 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:43.376025 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373903 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373906 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373908 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373913 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373916 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373919 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373922 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373925 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373929 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373932 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373934 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373937 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373940 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373943 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373945 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373948 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373951 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373955 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373959 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373962 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:43.376549 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373965 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373968 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373972 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373976 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373978 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373981 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373985 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373988 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373991 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373994 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.373996 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374000 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374003 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374005 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374008 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374011 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374016 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374019 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374023 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374026 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:43.377065 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374048 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374053 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374057 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374060 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374063 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374065 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374068 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374070 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374073 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374076 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374079 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374081 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374084 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374087 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374090 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374093 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374096 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374521 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374526 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:43.377583 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374530 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374533 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374537 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374541 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374544 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374548 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374552 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374556 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374559 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374563 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374566 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374569 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374571 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374574 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374577 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374580 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374583 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374586 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374589 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:43.378074 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374593 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374596 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374598 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374602 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374605 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374608 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374610 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374613 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374616 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374619 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374621 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374624 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374627 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374630 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374633 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374635 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374638 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374641 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374644 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374646 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:43.378640 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374649 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374652 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374655 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374658 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374660 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374664 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374666 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374669 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374672 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374674 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374677 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374680 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374683 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374686 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374689 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374692 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374695 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374698 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374700 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374703 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:43.379146 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374706 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374709 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374711 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374714 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374717 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374720 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374723 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374725 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374728 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374731 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374733 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374736 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374739 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374743 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374746 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374749 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374751 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374754 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374757 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:43.379685 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374760 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374762 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374765 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374768 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374774 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374778 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374781 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374784 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374786 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374789 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374792 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374795 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374798 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374800 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374803 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374807 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374811 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374814 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.374816 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374894 2585 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374901 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 11:15:43.380174 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374910 2585 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374917 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374924 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374929 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374936 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374941 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374945 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374948 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374952 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374955 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374959 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374962 2585 flags.go:64] FLAG: --cgroup-root="" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374965 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374969 2585 flags.go:64] FLAG: --client-ca-file="" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374972 2585 flags.go:64] FLAG: --cloud-provider="external" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374977 2585 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374984 2585 flags.go:64] FLAG: --cluster-domain="" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374987 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374991 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374994 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.374997 2585 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375001 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375004 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 11:15:43.380729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375008 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375011 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375014 2585 flags.go:64] FLAG: --contention-profiling="false" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375017 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375021 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375024 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375027 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375032 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375035 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375038 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375041 2585 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375044 2585 flags.go:64] FLAG: --enable-server="true" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375047 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375052 2585 flags.go:64] FLAG: --event-burst="100" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375055 2585 flags.go:64] FLAG: --event-qps="50" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375058 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375061 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375064 2585 flags.go:64] FLAG: --eviction-hard="" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375069 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375072 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375075 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375078 2585 flags.go:64] FLAG: --eviction-soft="" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375081 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375084 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375090 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 11:15:43.381309 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375098 2585 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375101 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375104 2585 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375107 2585 flags.go:64] FLAG: --feature-gates="" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375111 2585 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375114 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375117 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375121 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375124 2585 flags.go:64] FLAG: --healthz-port="10248" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375127 2585 flags.go:64] FLAG: --help="false" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375130 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375134 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375137 2585 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375140 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375143 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375147 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375150 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375153 2585 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375156 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375160 2585 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375163 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375166 2585 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375169 2585 flags.go:64] FLAG: --kube-reserved="" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375173 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 11:15:43.382007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375175 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375179 2585 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375182 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375185 2585 flags.go:64] FLAG: --lock-file="" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375188 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375191 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375194 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375201 2585 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375207 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375210 2585 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375213 2585 flags.go:64] FLAG: --logging-format="text" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375216 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375220 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375223 2585 flags.go:64] FLAG: --manifest-url="" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375225 2585 flags.go:64] FLAG: --manifest-url-header="" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375230 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375233 2585 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375237 2585 flags.go:64] FLAG: --max-pods="110" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375240 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375243 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375246 2585 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375249 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375253 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375256 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375259 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 11:15:43.382710 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375266 2585 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375269 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375273 2585 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375276 2585 flags.go:64] FLAG: --pod-cidr="" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375279 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375285 2585 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375288 2585 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375291 2585 flags.go:64] FLAG: --pods-per-core="0" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375294 2585 flags.go:64] FLAG: --port="10250" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375297 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375300 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00558afb7ea7ed414" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375304 2585 flags.go:64] FLAG: --qos-reserved="" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375307 2585 flags.go:64] FLAG: --read-only-port="10255" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375311 2585 flags.go:64] FLAG: --register-node="true" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375314 2585 flags.go:64] FLAG: --register-with-taints="" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375320 2585 flags.go:64] FLAG: --registry-burst="10" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375324 2585 flags.go:64] FLAG: --registry-qps="5" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375327 2585 flags.go:64] FLAG: --reserved-cpus="" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375331 2585 flags.go:64] FLAG: --reserved-memory="" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375335 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375338 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375341 2585 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375344 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375347 2585 flags.go:64] FLAG: --runonce="false" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375350 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 11:15:43.383365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375354 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375357 2585 flags.go:64] FLAG: --seccomp-default="false" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375360 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375364 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375367 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375370 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375373 2585 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375376 2585 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375379 2585 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375382 2585 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375385 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375388 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375391 2585 flags.go:64] FLAG: --system-cgroups="" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375394 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375400 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375403 2585 flags.go:64] FLAG: --tls-cert-file="" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375406 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375410 2585 flags.go:64] FLAG: --tls-min-version="" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375413 2585 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375416 2585 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375434 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375437 2585 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375443 2585 flags.go:64] FLAG: --v="2" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375448 2585 flags.go:64] FLAG: --version="false" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375453 2585 flags.go:64] FLAG: --vmodule="" Jul 07 11:15:43.384036 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375457 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375460 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375585 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375589 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375592 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375596 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375599 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375602 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375605 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375609 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375613 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375616 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375619 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375622 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375624 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375627 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375630 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375633 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375635 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375638 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:43.384704 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375641 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375644 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375647 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375649 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375652 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375655 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375658 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375660 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375664 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375670 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375673 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375677 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375680 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375683 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375686 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375688 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375691 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375694 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375697 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375699 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:43.385233 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375702 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375705 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375707 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375710 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375713 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375716 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375718 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375721 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375724 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375727 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375729 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375732 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375735 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375738 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375741 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375743 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375746 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375750 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375754 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375758 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:43.385801 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375760 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375767 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375770 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375773 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375776 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375781 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375785 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375788 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375791 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375794 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375797 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375800 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375802 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375805 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375808 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375811 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375813 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375816 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375819 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:43.386318 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375822 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375825 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375827 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375830 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375833 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375836 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375838 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375841 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375844 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375847 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375850 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375852 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375855 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375858 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375864 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375867 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375869 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375872 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375875 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375879 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:43.386822 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375882 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.375885 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.375893 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.383261 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.383280 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.383295 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.383305 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383379 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383386 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383389 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383392 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383395 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383398 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383404 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:43.387327 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383408 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383411 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383414 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383432 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383435 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383438 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383441 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383444 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383447 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383450 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383453 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383457 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383461 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383464 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383467 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383470 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383473 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383476 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383479 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:43.387800 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383482 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383485 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383488 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383492 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383496 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383499 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383502 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383505 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383507 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383510 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383513 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383515 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383518 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383521 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383524 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383527 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383529 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383532 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383535 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383538 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:43.388291 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383541 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383543 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383546 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383549 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383552 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383555 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383557 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383560 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383563 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383566 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383568 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383571 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383574 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383576 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383579 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383583 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383586 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383589 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383592 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383595 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:43.388814 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383597 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383600 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383603 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383606 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383608 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383611 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383614 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383617 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383620 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383624 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383628 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383631 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383634 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383637 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383640 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383643 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383646 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383648 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383651 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:43.389324 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383654 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383656 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383659 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383662 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383664 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383667 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383670 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383673 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383677 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383680 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383683 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383686 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383689 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383691 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.383697 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:15:43.389828 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383833 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383838 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383841 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383844 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383847 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383851 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383854 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383857 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383860 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383863 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383866 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383869 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383871 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383874 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383877 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383880 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383882 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383885 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383888 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383890 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:43.390224 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383893 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383896 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383899 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383902 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383907 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383911 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383914 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383917 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383921 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383924 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383926 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383929 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383932 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383935 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383937 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383940 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383942 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383945 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383948 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383950 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:43.390803 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383953 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383956 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383959 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383961 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383964 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383967 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383970 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383973 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383975 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383978 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383981 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383984 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383986 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383989 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383992 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383994 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.383997 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384000 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384004 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384006 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:43.391720 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384009 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384013 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384015 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384018 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384021 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384023 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384026 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384029 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384031 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384034 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384037 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384040 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384043 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384045 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384050 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384054 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384057 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384060 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384065 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:43.392511 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384068 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384072 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384075 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384078 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384081 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384083 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384087 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384089 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384092 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384095 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384099 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384102 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384105 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384107 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384110 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384114 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384117 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384119 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384122 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:43.393124 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:43.384125 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:43.393963 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.384130 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:15:43.393963 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.384247 2585 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 11:15:43.393963 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.386385 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 11:15:43.393963 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.388150 2585 server.go:1019] "Starting client certificate rotation" Jul 07 11:15:43.393963 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.388264 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 11:15:43.393963 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.388314 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 11:15:43.393963 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.393027 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 11:15:43.394852 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.394827 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 11:15:43.404796 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.404778 2585 log.go:25] "Validated CRI v1 runtime API" Jul 07 11:15:43.408187 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.408171 2585 log.go:25] "Validated CRI v1 image API" Jul 07 11:15:43.409735 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.409718 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 11:15:43.413152 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.413128 2585 fs.go:135] Filesystem UUIDs: map[2db38c17-c585-43fd-ac5e-800f494184f8:/dev/nvme0n1p3 731e5d80-db91-4cc3-98dc-8c6799ead375:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 07 11:15:43.413197 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.413154 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 11:15:43.418769 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.418655 2585 manager.go:216] Machine: {Timestamp:2026-07-07 11:15:43.417173279 +0000 UTC m=+1.005039763 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099749 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec236b0dc4f58b771b981af5480ede20 SystemUUID:ec236b0d-c4f5-8b77-1b98-1af5480ede20 BootID:0b39f10b-d42e-4586-9b45-129a79e95692 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ac:89:8d:3f:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ac:89:8d:3f:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:c1:ac:52:0e:77 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 11:15:43.418769 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.418765 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 11:15:43.418877 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.418850 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 11:15:43.420297 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.420283 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 11:15:43.421961 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.421930 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 11:15:43.422107 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.421965 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-166.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 11:15:43.423393 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.423381 2585 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 11:15:43.423441 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.423398 2585 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 11:15:43.423441 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.423406 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 11:15:43.424030 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.424019 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 11:15:43.425310 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.425300 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 11:15:43.428399 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.428388 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 07 11:15:43.428557 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.428546 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 11:15:43.428595 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.428581 2585 kubelet.go:486] "Attempting to sync node with API server" Jul 07 11:15:43.428625 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.428596 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 11:15:43.428625 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.428620 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 11:15:43.428679 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.428636 2585 kubelet.go:398] "Adding apiserver pod source" Jul 07 11:15:43.428679 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.428650 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 11:15:43.429121 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.429107 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 11:15:43.429177 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.429128 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 11:15:43.431373 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.431353 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 11:15:43.434735 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.434717 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 11:15:43.434811 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.434751 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 11:15:43.436076 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436057 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 11:15:43.436076 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436079 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 11:15:43.436181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436086 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 11:15:43.436181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436094 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 11:15:43.436181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436114 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 11:15:43.436181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436132 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 11:15:43.436181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436144 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 11:15:43.436181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436150 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 11:15:43.436181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436158 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 11:15:43.436181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436165 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 11:15:43.436181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436181 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 11:15:43.436181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.436187 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 11:15:43.437619 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.437601 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 11:15:43.437699 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.437635 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 11:15:43.441062 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.441044 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-166.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 11:15:43.441458 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.441439 2585 server.go:1268] "Started kubelet" Jul 07 11:15:43.441569 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.441491 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 11:15:43.441723 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.441663 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 11:15:43.441782 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.441746 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 11:15:43.441827 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.441701 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 11:15:43.441896 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.441711 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-166.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 11:15:43.442488 ip-10-0-135-166 systemd[1]: Started Kubernetes Kubelet. Jul 07 11:15:43.443128 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.442919 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 11:15:43.443712 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.443700 2585 server.go:310] "Adding debug handlers to kubelet server" Jul 07 11:15:43.447577 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.447559 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 11:15:43.449504 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.448617 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-166.ec2.internal.18bffdb7d41b2e4a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-166.ec2.internal,UID:ip-10-0-135-166.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-166.ec2.internal,},FirstTimestamp:2026-07-07 11:15:43.44139937 +0000 UTC m=+1.029265853,LastTimestamp:2026-07-07 11:15:43.44139937 +0000 UTC m=+1.029265853,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-166.ec2.internal,}" Jul 07 11:15:43.450691 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.450672 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 11:15:43.450691 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.450683 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 11:15:43.450887 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.450770 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-166.ec2.internal\" not found" Jul 07 11:15:43.451300 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451281 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 11:15:43.451399 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451284 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 11:15:43.451399 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451326 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 11:15:43.451518 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451436 2585 reconstruct.go:97] "Volume reconstruction finished" Jul 07 11:15:43.451518 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451445 2585 reconciler.go:29] "Reconciler: start to sync state" Jul 07 11:15:43.451646 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451616 2585 factory.go:55] Registering systemd factory Jul 07 11:15:43.451714 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451652 2585 factory.go:223] Registration of the systemd container factory successfully Jul 07 11:15:43.451880 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451861 2585 factory.go:153] Registering CRI-O factory Jul 07 11:15:43.451880 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451881 2585 factory.go:223] Registration of the crio container factory successfully Jul 07 11:15:43.452027 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451946 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 11:15:43.452027 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451973 2585 factory.go:103] Registering Raw factory Jul 07 11:15:43.452027 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.451988 2585 manager.go:1196] Started watching for new ooms in manager Jul 07 11:15:43.452471 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.452455 2585 manager.go:349] Starting recovery of all containers Jul 07 11:15:43.455259 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.455230 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 11:15:43.455380 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.455260 2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-166.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 11:15:43.465243 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.465224 2585 manager.go:354] Recovery completed Jul 07 11:15:43.469953 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.469935 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:43.471972 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.471947 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:43.472047 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.471986 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:43.472047 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.472000 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:43.472634 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.472618 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 11:15:43.472634 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.472635 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 11:15:43.472775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.472655 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 07 11:15:43.472775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.472621 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:43.473343 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.473329 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:43.473343 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.473355 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:43.473444 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.473365 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:43.474236 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.474222 2585 policy_none.go:49] "None policy: Start" Jul 07 11:15:43.474304 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.474242 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 11:15:43.474304 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.474257 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 11:15:43.474611 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.474537 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-166.ec2.internal.18bffdb7d5edac70 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-166.ec2.internal,UID:ip-10-0-135-166.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-166.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-166.ec2.internal,},FirstTimestamp:2026-07-07 11:15:43.47197144 +0000 UTC m=+1.059837926,LastTimestamp:2026-07-07 11:15:43.47197144 +0000 UTC m=+1.059837926,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-166.ec2.internal,}" Jul 07 11:15:43.475382 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.475369 2585 policy_none.go:47] "Start" Jul 07 11:15:43.478198 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.478179 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v4zpg" Jul 07 11:15:43.482577 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.482502 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-166.ec2.internal.18bffdb7d5edfb97 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-166.ec2.internal,UID:ip-10-0-135-166.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-166.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-166.ec2.internal,},FirstTimestamp:2026-07-07 11:15:43.471991703 +0000 UTC m=+1.059858191,LastTimestamp:2026-07-07 11:15:43.471991703 +0000 UTC m=+1.059858191,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-166.ec2.internal,}" Jul 07 11:15:43.486818 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.486797 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v4zpg" Jul 07 11:15:43.525012 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.524994 2585 manager.go:337] "Starting Device Plugin manager" Jul 07 11:15:43.525139 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.525036 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 11:15:43.525139 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.525049 2585 server.go:87] "Starting device plugin registration server" Jul 07 11:15:43.525335 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.525320 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 11:15:43.525403 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.525336 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 11:15:43.525466 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.525444 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 11:15:43.525545 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.525532 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 11:15:43.525592 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.525545 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 11:15:43.526243 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.526219 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 11:15:43.526314 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.526282 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-166.ec2.internal\" not found" Jul 07 11:15:43.627517 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.627422 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:43.628771 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.628751 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:43.628858 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.628796 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:43.628858 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.628815 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:43.628858 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.628855 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.637657 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.637630 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.637657 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.637654 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-166.ec2.internal\": node \"ip-10-0-135-166.ec2.internal\" not found" Jul 07 11:15:43.659418 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.659392 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-166.ec2.internal\" not found" Jul 07 11:15:43.667413 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.667382 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 11:15:43.668748 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.668729 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 11:15:43.668828 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.668780 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 11:15:43.668828 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.668805 2585 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 11:15:43.668896 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.668847 2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 11:15:43.671308 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.671293 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:43.759521 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.759481 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-166.ec2.internal\" not found" Jul 07 11:15:43.769356 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.769310 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-166.ec2.internal"] Jul 07 11:15:43.769445 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.769432 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:43.770486 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.770470 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:43.770554 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.770502 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:43.770554 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.770512 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:43.771873 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.771860 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:43.771981 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.771964 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.772042 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.772007 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:43.773267 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.773248 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:43.773382 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.773278 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:43.773382 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.773288 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:43.773382 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.773248 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:43.773382 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.773380 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:43.773516 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.773393 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:43.774840 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.774827 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.774885 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.774852 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:43.775622 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.775604 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:43.775701 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.775632 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:43.775701 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.775672 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:43.793021 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.793001 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-166.ec2.internal\" not found" node="ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.797432 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.797407 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-166.ec2.internal\" not found" node="ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.853321 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.853293 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/44b7522ba8d1c0a5a9cbf1723295ee5c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal\" (UID: \"44b7522ba8d1c0a5a9cbf1723295ee5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.853439 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.853325 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44b7522ba8d1c0a5a9cbf1723295ee5c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal\" (UID: \"44b7522ba8d1c0a5a9cbf1723295ee5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.853439 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.853342 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b14f96639c7f7d472ab32a017c1adf2c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-166.ec2.internal\" (UID: \"b14f96639c7f7d472ab32a017c1adf2c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.859841 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.859816 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-166.ec2.internal\" not found" Jul 07 11:15:43.954080 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.954018 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44b7522ba8d1c0a5a9cbf1723295ee5c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal\" (UID: \"44b7522ba8d1c0a5a9cbf1723295ee5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.954080 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.954056 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b14f96639c7f7d472ab32a017c1adf2c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-166.ec2.internal\" (UID: \"b14f96639c7f7d472ab32a017c1adf2c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.954080 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.954074 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/44b7522ba8d1c0a5a9cbf1723295ee5c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal\" (UID: \"44b7522ba8d1c0a5a9cbf1723295ee5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.954233 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.954112 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/44b7522ba8d1c0a5a9cbf1723295ee5c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal\" (UID: \"44b7522ba8d1c0a5a9cbf1723295ee5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.954233 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.954125 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b14f96639c7f7d472ab32a017c1adf2c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-166.ec2.internal\" (UID: \"b14f96639c7f7d472ab32a017c1adf2c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.954233 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:43.954116 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44b7522ba8d1c0a5a9cbf1723295ee5c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal\" (UID: \"44b7522ba8d1c0a5a9cbf1723295ee5c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" Jul 07 11:15:43.960648 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:43.960627 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-166.ec2.internal\" not found" Jul 07 11:15:44.061779 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.061734 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-166.ec2.internal\" not found" Jul 07 11:15:44.097614 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.097584 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" Jul 07 11:15:44.100178 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.100160 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-166.ec2.internal" Jul 07 11:15:44.162548 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.162506 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-166.ec2.internal\" not found" Jul 07 11:15:44.259982 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.259898 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:44.351328 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.351292 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" Jul 07 11:15:44.363622 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.363597 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 11:15:44.365783 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.365756 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-166.ec2.internal" Jul 07 11:15:44.375661 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.375641 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 11:15:44.387630 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.387608 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 11:15:44.387793 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.387771 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 11:15:44.387793 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.387777 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 11:15:44.429027 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.428999 2585 apiserver.go:52] "Watching apiserver" Jul 07 11:15:44.429256 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.429237 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:44.438165 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.438143 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 11:15:44.438567 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.438548 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-5zc94","kube-system/konnectivity-agent-qw6cr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal","openshift-multus/multus-9dcwb","openshift-multus/multus-additional-cni-plugins-rfs6d","openshift-multus/network-metrics-daemon-9rwbg","openshift-network-operator/iptables-alerter-9qxtj","kube-system/kube-apiserver-proxy-ip-10-0-135-166.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc","openshift-cluster-node-tuning-operator/tuned-pr79k","openshift-dns/node-resolver-kz98j","openshift-image-registry/node-ca-7bn6j","openshift-network-diagnostics/network-check-target-kzrqj","openshift-ovn-kubernetes/ovnkube-node-8sv4f"] Jul 07 11:15:44.440173 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.440159 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9qxtj" Jul 07 11:15:44.441408 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.441386 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:44.441502 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.441448 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:15:44.442680 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.442667 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.443472 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.443451 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:15:44.443565 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.443472 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dh2cr\"" Jul 07 11:15:44.443565 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.443532 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 11:15:44.443677 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.443603 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 11:15:44.444439 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.444423 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.445212 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.445191 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 11:15:44.445385 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.445371 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 11:15:44.445707 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.445691 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.446922 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.446844 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hj7dq\"" Jul 07 11:15:44.446922 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.446875 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 11:15:44.446922 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.446879 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 11:15:44.446922 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.446912 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 11:15:44.447181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.446857 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 11:15:44.447181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.447066 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:44.447181 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.447118 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:15:44.447324 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.447234 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 11:15:44.447324 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.447282 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 11:15:44.448181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.448157 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 11:15:44.448291 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.448186 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tcpsg\"" Jul 07 11:15:44.448291 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.448154 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 11:15:44.448414 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.448304 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 11:15:44.448414 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.448322 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 11:15:44.448414 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.448327 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-w2frx\"" Jul 07 11:15:44.448592 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.448557 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.449954 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.449934 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:44.450049 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.449983 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:15:44.450764 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.450743 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 11:15:44.451353 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.451192 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 11:15:44.451353 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.451236 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 11:15:44.451353 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.451284 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:15:44.451850 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.451664 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ljtwf\"" Jul 07 11:15:44.451958 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.451938 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 11:15:44.454368 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.454342 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 11:15:44.454734 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.454716 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 11:15:44.455093 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.455063 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.455488 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.455466 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fpz7v\"" Jul 07 11:15:44.457028 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457008 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-daemon-config\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.457092 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457036 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkkb5\" (UniqueName: \"kubernetes.io/projected/a648f49b-76f9-44b9-9165-d96f40f7f836-kube-api-access-fkkb5\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.457092 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457052 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-run\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.457092 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457067 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-sys\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.457092 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457084 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-run-ovn-kubernetes\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.457250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457140 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9609fab6-8573-4b1d-8a7b-add43d375ecc-ovnkube-script-lib\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.457250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457163 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-system-cni-dir\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.457250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457181 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-run-netns\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.457250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457197 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-cnibin\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.457250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457220 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-slash\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.457250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457236 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-cni-netd\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.457250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457251 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-conf-dir\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.457591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457286 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-registration-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.457591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457320 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-sysconfig\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.457591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457346 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccbjg\" (UniqueName: \"kubernetes.io/projected/19149fd0-add1-41dc-bfde-eda95fea0859-kube-api-access-ccbjg\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.457591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457401 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kz98j" Jul 07 11:15:44.457591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457403 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-kubelet\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.457591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457533 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-run-openvswitch\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.457591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457557 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9609fab6-8573-4b1d-8a7b-add43d375ecc-ovnkube-config\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.457591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457585 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-649fn\" (UniqueName: \"kubernetes.io/projected/0c92ffd4-b8cb-4485-827e-206cf7e95421-kube-api-access-649fn\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.457882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457615 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.457882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457645 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-socket-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.457882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457687 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-sysctl-conf\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.457882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457721 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/19149fd0-add1-41dc-bfde-eda95fea0859-etc-tuned\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.457882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457751 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-systemd-units\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.457882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457787 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxrqk\" (UniqueName: \"kubernetes.io/projected/9609fab6-8573-4b1d-8a7b-add43d375ecc-kube-api-access-xxrqk\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.457882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457811 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-os-release\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.457882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457827 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-run-netns\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.457882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457841 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-etc-kubernetes\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.457882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457868 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45bf26b3-aacb-4588-bf31-04a417518c64-kubelet-config\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457899 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c92ffd4-b8cb-4485-827e-206cf7e95421-cni-binary-copy\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457921 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c92ffd4-b8cb-4485-827e-206cf7e95421-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457938 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19149fd0-add1-41dc-bfde-eda95fea0859-tmp\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457964 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-var-lib-openvswitch\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.457990 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-cni-dir\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458015 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-run-multus-certs\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458042 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-kubernetes\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458098 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-host\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458114 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458125 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-cni-bin\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458150 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2bd8b32a-fafe-400a-8f05-34f364bf41dd-iptables-alerter-script\") pod \"iptables-alerter-9qxtj\" (UID: \"2bd8b32a-fafe-400a-8f05-34f364bf41dd\") " pod="openshift-network-operator/iptables-alerter-9qxtj" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458155 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458176 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458200 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-sysctl-d\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.458240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458226 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-etc-openvswitch\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458252 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-log-socket\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458275 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-cnibin\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458298 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-hostroot\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458322 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c29cf0f1-4400-493b-8746-91f03ba47db6-agent-certs\") pod \"konnectivity-agent-qw6cr\" (UID: \"c29cf0f1-4400-493b-8746-91f03ba47db6\") " pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458347 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2nqkf\"" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458363 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458388 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-device-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458404 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p85hw\" (UniqueName: \"kubernetes.io/projected/bc7950df-04e9-423e-9390-0e5f881efe4a-kube-api-access-p85hw\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458420 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458435 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-systemd\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458452 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-run-ovn\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458474 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-node-log\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458496 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9609fab6-8573-4b1d-8a7b-add43d375ecc-ovn-node-metrics-cert\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458513 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-socket-dir-parent\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458528 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bd8b32a-fafe-400a-8f05-34f364bf41dd-host-slash\") pod \"iptables-alerter-9qxtj\" (UID: \"2bd8b32a-fafe-400a-8f05-34f364bf41dd\") " pod="openshift-network-operator/iptables-alerter-9qxtj" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458545 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.458737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458574 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9609fab6-8573-4b1d-8a7b-add43d375ecc-env-overrides\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458600 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a648f49b-76f9-44b9-9165-d96f40f7f836-cni-binary-copy\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458618 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c29cf0f1-4400-493b-8746-91f03ba47db6-konnectivity-ca\") pod \"konnectivity-agent-qw6cr\" (UID: \"c29cf0f1-4400-493b-8746-91f03ba47db6\") " pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458635 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-os-release\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458657 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c92ffd4-b8cb-4485-827e-206cf7e95421-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458673 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-sys-fs\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458693 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45bf26b3-aacb-4588-bf31-04a417518c64-dbus\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458718 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-system-cni-dir\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458761 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cjbm\" (UniqueName: \"kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm\") pod \"network-check-target-kzrqj\" (UID: \"617abf3b-a72a-4600-8764-60aab5668bcd\") " pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458797 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvwhz\" (UniqueName: \"kubernetes.io/projected/2bd8b32a-fafe-400a-8f05-34f364bf41dd-kube-api-access-cvwhz\") pod \"iptables-alerter-9qxtj\" (UID: \"2bd8b32a-fafe-400a-8f05-34f364bf41dd\") " pod="openshift-network-operator/iptables-alerter-9qxtj" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458813 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7bn6j" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458822 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-run-k8s-cni-cncf-io\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458847 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-var-lib-cni-multus\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458891 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-var-lib-kubelet\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458912 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-etc-selinux\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458935 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-var-lib-kubelet\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.459268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.458997 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdz7b\" (UniqueName: \"kubernetes.io/projected/5da61a2d-4113-4869-99bf-f0d24e5edf97-kube-api-access-sdz7b\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:44.459754 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.459034 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-modprobe-d\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.459754 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.459054 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-lib-modules\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.459754 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.459069 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-run-systemd\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.459754 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.459085 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-var-lib-cni-bin\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.459893 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.459816 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 11:15:44.459939 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.459916 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 11:15:44.460047 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.460022 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xw8q6\"" Jul 07 11:15:44.461367 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.461349 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 11:15:44.461478 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.461384 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 11:15:44.461478 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.461441 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 11:15:44.461478 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.461464 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pbvtp\"" Jul 07 11:15:44.468463 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.468397 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 11:15:44.490108 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.490058 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 11:10:44 +0000 UTC" deadline="2028-01-21 15:10:02.088158967 +0000 UTC" Jul 07 11:15:44.490181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.490107 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13515h54m17.598054539s" Jul 07 11:15:44.497607 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.497581 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qsjzm" Jul 07 11:15:44.505880 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.505859 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qsjzm" Jul 07 11:15:44.518559 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.518497 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:44.552370 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.552350 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.559619 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkkb5\" (UniqueName: \"kubernetes.io/projected/a648f49b-76f9-44b9-9165-d96f40f7f836-kube-api-access-fkkb5\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.559667 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-run\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.559709 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-sys\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.559743 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-run-ovn-kubernetes\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.559769 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9609fab6-8573-4b1d-8a7b-add43d375ecc-ovnkube-script-lib\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.559803 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef118293-20df-4a0e-8b0f-9204c6096441-host\") pod \"node-ca-7bn6j\" (UID: \"ef118293-20df-4a0e-8b0f-9204c6096441\") " pod="openshift-image-registry/node-ca-7bn6j" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.559834 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-system-cni-dir\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.559865 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-run-netns\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.559964 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-cnibin\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560001 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-slash\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560107 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-cni-netd\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560178 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-conf-dir\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560221 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-registration-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560252 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-sysconfig\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560305 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccbjg\" (UniqueName: \"kubernetes.io/projected/19149fd0-add1-41dc-bfde-eda95fea0859-kube-api-access-ccbjg\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560339 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-kubelet\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.562132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560388 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-run-openvswitch\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560442 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9609fab6-8573-4b1d-8a7b-add43d375ecc-ovnkube-config\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560486 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-649fn\" (UniqueName: \"kubernetes.io/projected/0c92ffd4-b8cb-4485-827e-206cf7e95421-kube-api-access-649fn\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560560 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560629 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-socket-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560687 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-sysctl-conf\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560724 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/19149fd0-add1-41dc-bfde-eda95fea0859-etc-tuned\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560771 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-systemd-units\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560803 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxrqk\" (UniqueName: \"kubernetes.io/projected/9609fab6-8573-4b1d-8a7b-add43d375ecc-kube-api-access-xxrqk\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560849 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-os-release\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560880 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-run-netns\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560918 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-slash\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.560930 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-etc-kubernetes\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561032 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-cni-netd\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561048 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45bf26b3-aacb-4588-bf31-04a417518c64-kubelet-config\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561084 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c92ffd4-b8cb-4485-827e-206cf7e95421-cni-binary-copy\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561120 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-registration-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.563072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561110 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c92ffd4-b8cb-4485-827e-206cf7e95421-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19149fd0-add1-41dc-bfde-eda95fea0859-tmp\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561247 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-var-lib-openvswitch\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561288 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-cni-dir\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561319 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-run-multus-certs\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561396 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-kubernetes\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561471 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-host\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561504 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-cni-bin\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561551 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brm4c\" (UniqueName: \"kubernetes.io/projected/ef118293-20df-4a0e-8b0f-9204c6096441-kube-api-access-brm4c\") pod \"node-ca-7bn6j\" (UID: \"ef118293-20df-4a0e-8b0f-9204c6096441\") " pod="openshift-image-registry/node-ca-7bn6j" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561610 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2bd8b32a-fafe-400a-8f05-34f364bf41dd-iptables-alerter-script\") pod \"iptables-alerter-9qxtj\" (UID: \"2bd8b32a-fafe-400a-8f05-34f364bf41dd\") " pod="openshift-network-operator/iptables-alerter-9qxtj" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561645 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561686 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-sysctl-d\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561716 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-etc-openvswitch\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561754 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-log-socket\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561780 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e2bee60-3e0b-403d-80a0-86946959866d-hosts-file\") pod \"node-resolver-kz98j\" (UID: \"3e2bee60-3e0b-403d-80a0-86946959866d\") " pod="openshift-dns/node-resolver-kz98j" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561810 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-cnibin\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561832 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-run-multus-certs\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562194 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-sysconfig\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.564250 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561970 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-host\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562255 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-cnibin\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562082 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-sys\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562058 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-cni-bin\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562272 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-log-socket\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562104 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-kubernetes\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562110 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-run-ovn-kubernetes\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562183 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562210 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-system-cni-dir\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562215 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-etc-openvswitch\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562168 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-cnibin\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562016 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c92ffd4-b8cb-4485-827e-206cf7e95421-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562463 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.561850 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-hostroot\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562582 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c29cf0f1-4400-493b-8746-91f03ba47db6-agent-certs\") pod \"konnectivity-agent-qw6cr\" (UID: \"c29cf0f1-4400-493b-8746-91f03ba47db6\") " pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562574 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-run-netns\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562630 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562612 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-hostroot\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.565066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562754 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-var-lib-openvswitch\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562758 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-kubelet\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562785 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-cni-dir\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562821 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2bd8b32a-fafe-400a-8f05-34f364bf41dd-iptables-alerter-script\") pod \"iptables-alerter-9qxtj\" (UID: \"2bd8b32a-fafe-400a-8f05-34f364bf41dd\") " pod="openshift-network-operator/iptables-alerter-9qxtj" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562869 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-socket-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562831 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-run-openvswitch\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562943 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-sysctl-d\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-device-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.562982 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-device-dir\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563005 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-conf-dir\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563041 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9609fab6-8573-4b1d-8a7b-add43d375ecc-ovnkube-config\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563110 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-run-netns\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563161 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-os-release\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563176 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-systemd-units\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563173 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563255 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p85hw\" (UniqueName: \"kubernetes.io/projected/bc7950df-04e9-423e-9390-0e5f881efe4a-kube-api-access-p85hw\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563306 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45bf26b3-aacb-4588-bf31-04a417518c64-kubelet-config\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563374 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-run\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.565962 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563446 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563471 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-etc-kubernetes\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563567 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-systemd\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563640 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-run-ovn\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.563653 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563676 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-node-log\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.563836 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret podName:45bf26b3-aacb-4588-bf31-04a417518c64 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:45.063803017 +0000 UTC m=+2.651669502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret") pod "global-pull-secret-syncer-5zc94" (UID: "45bf26b3-aacb-4588-bf31-04a417518c64") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563857 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-sysctl-conf\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563868 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-node-log\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563909 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9609fab6-8573-4b1d-8a7b-add43d375ecc-ovn-node-metrics-cert\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563915 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-run-ovn\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563959 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-systemd\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.563987 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-socket-dir-parent\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564025 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c92ffd4-b8cb-4485-827e-206cf7e95421-cni-binary-copy\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564038 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9609fab6-8573-4b1d-8a7b-add43d375ecc-ovnkube-script-lib\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564081 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bd8b32a-fafe-400a-8f05-34f364bf41dd-host-slash\") pod \"iptables-alerter-9qxtj\" (UID: \"2bd8b32a-fafe-400a-8f05-34f364bf41dd\") " pod="openshift-network-operator/iptables-alerter-9qxtj" Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.564113 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:44.566876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564188 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564544 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564561 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bd8b32a-fafe-400a-8f05-34f364bf41dd-host-slash\") pod \"iptables-alerter-9qxtj\" (UID: \"2bd8b32a-fafe-400a-8f05-34f364bf41dd\") " pod="openshift-network-operator/iptables-alerter-9qxtj" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564560 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-socket-dir-parent\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.564721 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs podName:5da61a2d-4113-4869-99bf-f0d24e5edf97 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:45.06470014 +0000 UTC m=+2.652566623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs") pod "network-metrics-daemon-9rwbg" (UID: "5da61a2d-4113-4869-99bf-f0d24e5edf97") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564809 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9609fab6-8573-4b1d-8a7b-add43d375ecc-env-overrides\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564838 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a648f49b-76f9-44b9-9165-d96f40f7f836-cni-binary-copy\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564865 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c29cf0f1-4400-493b-8746-91f03ba47db6-konnectivity-ca\") pod \"konnectivity-agent-qw6cr\" (UID: \"c29cf0f1-4400-493b-8746-91f03ba47db6\") " pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564894 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-os-release\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564921 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c92ffd4-b8cb-4485-827e-206cf7e95421-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564949 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-sys-fs\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.564972 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45bf26b3-aacb-4588-bf31-04a417518c64-dbus\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565001 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-system-cni-dir\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565029 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cjbm\" (UniqueName: \"kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm\") pod \"network-check-target-kzrqj\" (UID: \"617abf3b-a72a-4600-8764-60aab5668bcd\") " pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565061 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvwhz\" (UniqueName: \"kubernetes.io/projected/2bd8b32a-fafe-400a-8f05-34f364bf41dd-kube-api-access-cvwhz\") pod \"iptables-alerter-9qxtj\" (UID: \"2bd8b32a-fafe-400a-8f05-34f364bf41dd\") " pod="openshift-network-operator/iptables-alerter-9qxtj" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565090 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-run-k8s-cni-cncf-io\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.567763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565115 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-var-lib-cni-multus\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565142 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-var-lib-kubelet\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565167 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-etc-selinux\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565192 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-var-lib-kubelet\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565219 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdz7b\" (UniqueName: \"kubernetes.io/projected/5da61a2d-4113-4869-99bf-f0d24e5edf97-kube-api-access-sdz7b\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565247 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-modprobe-d\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565287 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-lib-modules\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565315 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-run-systemd\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565352 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef118293-20df-4a0e-8b0f-9204c6096441-serviceca\") pod \"node-ca-7bn6j\" (UID: \"ef118293-20df-4a0e-8b0f-9204c6096441\") " pod="openshift-image-registry/node-ca-7bn6j" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565383 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e2bee60-3e0b-403d-80a0-86946959866d-tmp-dir\") pod \"node-resolver-kz98j\" (UID: \"3e2bee60-3e0b-403d-80a0-86946959866d\") " pod="openshift-dns/node-resolver-kz98j" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565412 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndw56\" (UniqueName: \"kubernetes.io/projected/3e2bee60-3e0b-403d-80a0-86946959866d-kube-api-access-ndw56\") pod \"node-resolver-kz98j\" (UID: \"3e2bee60-3e0b-403d-80a0-86946959866d\") " pod="openshift-dns/node-resolver-kz98j" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565459 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-var-lib-cni-bin\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565505 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-daemon-config\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565521 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c29cf0f1-4400-493b-8746-91f03ba47db6-konnectivity-ca\") pod \"konnectivity-agent-qw6cr\" (UID: \"c29cf0f1-4400-493b-8746-91f03ba47db6\") " pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565554 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-var-lib-kubelet\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565584 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-system-cni-dir\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565601 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-etc-selinux\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.568601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565651 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-var-lib-kubelet\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565664 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-run-k8s-cni-cncf-io\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565712 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-var-lib-cni-multus\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565767 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9609fab6-8573-4b1d-8a7b-add43d375ecc-run-systemd\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565859 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9609fab6-8573-4b1d-8a7b-add43d375ecc-env-overrides\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.565915 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c92ffd4-b8cb-4485-827e-206cf7e95421-os-release\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.566002 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-etc-modprobe-d\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.566030 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a648f49b-76f9-44b9-9165-d96f40f7f836-cni-binary-copy\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.566078 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc7950df-04e9-423e-9390-0e5f881efe4a-sys-fs\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.566087 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19149fd0-add1-41dc-bfde-eda95fea0859-lib-modules\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.566103 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a648f49b-76f9-44b9-9165-d96f40f7f836-host-var-lib-cni-bin\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.566156 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a648f49b-76f9-44b9-9165-d96f40f7f836-multus-daemon-config\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.566185 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45bf26b3-aacb-4588-bf31-04a417518c64-dbus\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.566361 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c92ffd4-b8cb-4485-827e-206cf7e95421-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.568084 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19149fd0-add1-41dc-bfde-eda95fea0859-tmp\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.568214 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9609fab6-8573-4b1d-8a7b-add43d375ecc-ovn-node-metrics-cert\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.568838 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/19149fd0-add1-41dc-bfde-eda95fea0859-etc-tuned\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.569514 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.568854 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c29cf0f1-4400-493b-8746-91f03ba47db6-agent-certs\") pod \"konnectivity-agent-qw6cr\" (UID: \"c29cf0f1-4400-493b-8746-91f03ba47db6\") " pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:15:44.571648 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.571626 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-649fn\" (UniqueName: \"kubernetes.io/projected/0c92ffd4-b8cb-4485-827e-206cf7e95421-kube-api-access-649fn\") pod \"multus-additional-cni-plugins-rfs6d\" (UID: \"0c92ffd4-b8cb-4485-827e-206cf7e95421\") " pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.572216 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.572194 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccbjg\" (UniqueName: \"kubernetes.io/projected/19149fd0-add1-41dc-bfde-eda95fea0859-kube-api-access-ccbjg\") pod \"tuned-pr79k\" (UID: \"19149fd0-add1-41dc-bfde-eda95fea0859\") " pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.574498 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.574478 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:15:44.574581 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.574502 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:15:44.574581 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.574516 2585 projected.go:196] Error preparing data for projected volume kube-api-access-5cjbm for pod openshift-network-diagnostics/network-check-target-kzrqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:44.574751 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:44.574594 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm podName:617abf3b-a72a-4600-8764-60aab5668bcd nodeName:}" failed. No retries permitted until 2026-07-07 11:15:45.074570709 +0000 UTC m=+2.662437186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5cjbm" (UniqueName: "kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm") pod "network-check-target-kzrqj" (UID: "617abf3b-a72a-4600-8764-60aab5668bcd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:44.574892 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.574872 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkkb5\" (UniqueName: \"kubernetes.io/projected/a648f49b-76f9-44b9-9165-d96f40f7f836-kube-api-access-fkkb5\") pod \"multus-9dcwb\" (UID: \"a648f49b-76f9-44b9-9165-d96f40f7f836\") " pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.575955 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.575934 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p85hw\" (UniqueName: \"kubernetes.io/projected/bc7950df-04e9-423e-9390-0e5f881efe4a-kube-api-access-p85hw\") pod \"aws-ebs-csi-driver-node-k8mfc\" (UID: \"bc7950df-04e9-423e-9390-0e5f881efe4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.577587 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.577562 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvwhz\" (UniqueName: \"kubernetes.io/projected/2bd8b32a-fafe-400a-8f05-34f364bf41dd-kube-api-access-cvwhz\") pod \"iptables-alerter-9qxtj\" (UID: \"2bd8b32a-fafe-400a-8f05-34f364bf41dd\") " pod="openshift-network-operator/iptables-alerter-9qxtj" Jul 07 11:15:44.577867 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.577848 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxrqk\" (UniqueName: \"kubernetes.io/projected/9609fab6-8573-4b1d-8a7b-add43d375ecc-kube-api-access-xxrqk\") pod \"ovnkube-node-8sv4f\" (UID: \"9609fab6-8573-4b1d-8a7b-add43d375ecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.578869 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.578844 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdz7b\" (UniqueName: \"kubernetes.io/projected/5da61a2d-4113-4869-99bf-f0d24e5edf97-kube-api-access-sdz7b\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:44.666573 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.666523 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef118293-20df-4a0e-8b0f-9204c6096441-serviceca\") pod \"node-ca-7bn6j\" (UID: \"ef118293-20df-4a0e-8b0f-9204c6096441\") " pod="openshift-image-registry/node-ca-7bn6j" Jul 07 11:15:44.666573 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.666577 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e2bee60-3e0b-403d-80a0-86946959866d-tmp-dir\") pod \"node-resolver-kz98j\" (UID: \"3e2bee60-3e0b-403d-80a0-86946959866d\") " pod="openshift-dns/node-resolver-kz98j" Jul 07 11:15:44.666847 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.666596 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndw56\" (UniqueName: \"kubernetes.io/projected/3e2bee60-3e0b-403d-80a0-86946959866d-kube-api-access-ndw56\") pod \"node-resolver-kz98j\" (UID: \"3e2bee60-3e0b-403d-80a0-86946959866d\") " pod="openshift-dns/node-resolver-kz98j" Jul 07 11:15:44.666847 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.666620 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef118293-20df-4a0e-8b0f-9204c6096441-host\") pod \"node-ca-7bn6j\" (UID: \"ef118293-20df-4a0e-8b0f-9204c6096441\") " pod="openshift-image-registry/node-ca-7bn6j" Jul 07 11:15:44.666847 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.666687 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brm4c\" (UniqueName: \"kubernetes.io/projected/ef118293-20df-4a0e-8b0f-9204c6096441-kube-api-access-brm4c\") pod \"node-ca-7bn6j\" (UID: \"ef118293-20df-4a0e-8b0f-9204c6096441\") " pod="openshift-image-registry/node-ca-7bn6j" Jul 07 11:15:44.666847 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.666707 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e2bee60-3e0b-403d-80a0-86946959866d-hosts-file\") pod \"node-resolver-kz98j\" (UID: \"3e2bee60-3e0b-403d-80a0-86946959866d\") " pod="openshift-dns/node-resolver-kz98j" Jul 07 11:15:44.666847 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.666790 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef118293-20df-4a0e-8b0f-9204c6096441-host\") pod \"node-ca-7bn6j\" (UID: \"ef118293-20df-4a0e-8b0f-9204c6096441\") " pod="openshift-image-registry/node-ca-7bn6j" Jul 07 11:15:44.666847 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.666837 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e2bee60-3e0b-403d-80a0-86946959866d-hosts-file\") pod \"node-resolver-kz98j\" (UID: \"3e2bee60-3e0b-403d-80a0-86946959866d\") " pod="openshift-dns/node-resolver-kz98j" Jul 07 11:15:44.667151 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.666995 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e2bee60-3e0b-403d-80a0-86946959866d-tmp-dir\") pod \"node-resolver-kz98j\" (UID: \"3e2bee60-3e0b-403d-80a0-86946959866d\") " pod="openshift-dns/node-resolver-kz98j" Jul 07 11:15:44.667151 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.667132 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef118293-20df-4a0e-8b0f-9204c6096441-serviceca\") pod \"node-ca-7bn6j\" (UID: \"ef118293-20df-4a0e-8b0f-9204c6096441\") " pod="openshift-image-registry/node-ca-7bn6j" Jul 07 11:15:44.678564 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.678531 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brm4c\" (UniqueName: \"kubernetes.io/projected/ef118293-20df-4a0e-8b0f-9204c6096441-kube-api-access-brm4c\") pod \"node-ca-7bn6j\" (UID: \"ef118293-20df-4a0e-8b0f-9204c6096441\") " pod="openshift-image-registry/node-ca-7bn6j" Jul 07 11:15:44.678770 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.678746 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndw56\" (UniqueName: \"kubernetes.io/projected/3e2bee60-3e0b-403d-80a0-86946959866d-kube-api-access-ndw56\") pod \"node-resolver-kz98j\" (UID: \"3e2bee60-3e0b-403d-80a0-86946959866d\") " pod="openshift-dns/node-resolver-kz98j" Jul 07 11:15:44.769100 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.769009 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9qxtj" Jul 07 11:15:44.785261 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.785236 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:15:44.801353 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.801328 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9dcwb" Jul 07 11:15:44.806187 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.806165 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rfs6d" Jul 07 11:15:44.823451 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.823434 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" Jul 07 11:15:44.829744 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:44.829709 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44b7522ba8d1c0a5a9cbf1723295ee5c.slice/crio-adb4de71f856ad2f80b491da9f6ee81b1cbd27de0259570fd63eb88b6f9e7641 WatchSource:0}: Error finding container adb4de71f856ad2f80b491da9f6ee81b1cbd27de0259570fd63eb88b6f9e7641: Status 404 returned error can't find the container with id adb4de71f856ad2f80b491da9f6ee81b1cbd27de0259570fd63eb88b6f9e7641 Jul 07 11:15:44.830084 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:44.830062 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c92ffd4_b8cb_4485_827e_206cf7e95421.slice/crio-309f13fbd6dbd9d800f32782f5bd207ca3755247e43eb45bc59b04c2a5d93af8 WatchSource:0}: Error finding container 309f13fbd6dbd9d800f32782f5bd207ca3755247e43eb45bc59b04c2a5d93af8: Status 404 returned error can't find the container with id 309f13fbd6dbd9d800f32782f5bd207ca3755247e43eb45bc59b04c2a5d93af8 Jul 07 11:15:44.830805 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:44.830781 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14f96639c7f7d472ab32a017c1adf2c.slice/crio-5bcf19c9670391f96130b4aea0c08de35849a764ef9ea4bcee9aaf01bdbf193c WatchSource:0}: Error finding container 5bcf19c9670391f96130b4aea0c08de35849a764ef9ea4bcee9aaf01bdbf193c: Status 404 returned error can't find the container with id 5bcf19c9670391f96130b4aea0c08de35849a764ef9ea4bcee9aaf01bdbf193c Jul 07 11:15:44.833165 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:44.833140 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc7950df_04e9_423e_9390_0e5f881efe4a.slice/crio-f8861e9ecc1fcb13a56d55629d96506ab197a8b86a8f2a60bc9874bdc623f282 WatchSource:0}: Error finding container f8861e9ecc1fcb13a56d55629d96506ab197a8b86a8f2a60bc9874bdc623f282: Status 404 returned error can't find the container with id f8861e9ecc1fcb13a56d55629d96506ab197a8b86a8f2a60bc9874bdc623f282 Jul 07 11:15:44.836898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.836883 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:15:44.840328 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.840263 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:15:44.848295 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:44.848276 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29cf0f1_4400_493b_8746_91f03ba47db6.slice/crio-378fca676801e757ae016a2c24a7471058675e3726d84312a491c83b6a04405e WatchSource:0}: Error finding container 378fca676801e757ae016a2c24a7471058675e3726d84312a491c83b6a04405e: Status 404 returned error can't find the container with id 378fca676801e757ae016a2c24a7471058675e3726d84312a491c83b6a04405e Jul 07 11:15:44.855540 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.855520 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pr79k" Jul 07 11:15:44.862288 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:44.862265 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19149fd0_add1_41dc_bfde_eda95fea0859.slice/crio-12d7abe0c414df3408647655671f4e57b0800900064066c273befd392c197d71 WatchSource:0}: Error finding container 12d7abe0c414df3408647655671f4e57b0800900064066c273befd392c197d71: Status 404 returned error can't find the container with id 12d7abe0c414df3408647655671f4e57b0800900064066c273befd392c197d71 Jul 07 11:15:44.888047 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.888017 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kz98j" Jul 07 11:15:44.892201 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:44.892175 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7bn6j" Jul 07 11:15:44.895795 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:44.895764 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2bee60_3e0b_403d_80a0_86946959866d.slice/crio-c6cb865dab236e9c80afb32e0ee10ef690f0b84c90b8915a431ed0126a85fe49 WatchSource:0}: Error finding container c6cb865dab236e9c80afb32e0ee10ef690f0b84c90b8915a431ed0126a85fe49: Status 404 returned error can't find the container with id c6cb865dab236e9c80afb32e0ee10ef690f0b84c90b8915a431ed0126a85fe49 Jul 07 11:15:44.900323 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:44.900302 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef118293_20df_4a0e_8b0f_9204c6096441.slice/crio-28d6e6aeba5d56d1df7e477316cce62bd3d31193b58fdd690c1ec405c3c10490 WatchSource:0}: Error finding container 28d6e6aeba5d56d1df7e477316cce62bd3d31193b58fdd690c1ec405c3c10490: Status 404 returned error can't find the container with id 28d6e6aeba5d56d1df7e477316cce62bd3d31193b58fdd690c1ec405c3c10490 Jul 07 11:15:45.070739 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.070653 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:45.070739 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.070690 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:45.070965 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:45.070827 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:45.070965 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:45.070834 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:45.070965 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:45.070884 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret podName:45bf26b3-aacb-4588-bf31-04a417518c64 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:46.070870094 +0000 UTC m=+3.658736577 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret") pod "global-pull-secret-syncer-5zc94" (UID: "45bf26b3-aacb-4588-bf31-04a417518c64") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:45.070965 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:45.070909 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs podName:5da61a2d-4113-4869-99bf-f0d24e5edf97 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:46.070890636 +0000 UTC m=+3.658757107 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs") pod "network-metrics-daemon-9rwbg" (UID: "5da61a2d-4113-4869-99bf-f0d24e5edf97") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:45.171256 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.171218 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cjbm\" (UniqueName: \"kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm\") pod \"network-check-target-kzrqj\" (UID: \"617abf3b-a72a-4600-8764-60aab5668bcd\") " pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:45.171468 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:45.171448 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:15:45.171518 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:45.171475 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:15:45.171518 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:45.171485 2585 projected.go:196] Error preparing data for projected volume kube-api-access-5cjbm for pod openshift-network-diagnostics/network-check-target-kzrqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:45.171584 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:45.171552 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm podName:617abf3b-a72a-4600-8764-60aab5668bcd nodeName:}" failed. No retries permitted until 2026-07-07 11:15:46.171537929 +0000 UTC m=+3.759404403 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cjbm" (UniqueName: "kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm") pod "network-check-target-kzrqj" (UID: "617abf3b-a72a-4600-8764-60aab5668bcd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:45.507961 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.507700 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:10:45 +0000 UTC" deadline="2028-01-23 18:24:42.957004855 +0000 UTC" Jul 07 11:15:45.508511 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.507963 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13567h8m57.449048586s" Jul 07 11:15:45.599393 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:45.599298 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9609fab6_8573_4b1d_8a7b_add43d375ecc.slice/crio-0baf442e21d546ee2a608023c283fe5a3aa2f8559d609959bca8b8d45d4f9462 WatchSource:0}: Error finding container 0baf442e21d546ee2a608023c283fe5a3aa2f8559d609959bca8b8d45d4f9462: Status 404 returned error can't find the container with id 0baf442e21d546ee2a608023c283fe5a3aa2f8559d609959bca8b8d45d4f9462 Jul 07 11:15:45.671520 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.670415 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:45.671520 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:45.670562 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:15:45.681288 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:15:45.681256 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda648f49b_76f9_44b9_9165_d96f40f7f836.slice/crio-ae8d5f0aa4ae6c9097c786587a0b1a11ad851dbdc9c6c809ccca8b2bca9c8e19 WatchSource:0}: Error finding container ae8d5f0aa4ae6c9097c786587a0b1a11ad851dbdc9c6c809ccca8b2bca9c8e19: Status 404 returned error can't find the container with id ae8d5f0aa4ae6c9097c786587a0b1a11ad851dbdc9c6c809ccca8b2bca9c8e19 Jul 07 11:15:45.682883 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.682805 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" event={"ID":"9609fab6-8573-4b1d-8a7b-add43d375ecc","Type":"ContainerStarted","Data":"0baf442e21d546ee2a608023c283fe5a3aa2f8559d609959bca8b8d45d4f9462"} Jul 07 11:15:45.691944 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.691907 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7bn6j" event={"ID":"ef118293-20df-4a0e-8b0f-9204c6096441","Type":"ContainerStarted","Data":"28d6e6aeba5d56d1df7e477316cce62bd3d31193b58fdd690c1ec405c3c10490"} Jul 07 11:15:45.698520 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.698474 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kz98j" event={"ID":"3e2bee60-3e0b-403d-80a0-86946959866d","Type":"ContainerStarted","Data":"c6cb865dab236e9c80afb32e0ee10ef690f0b84c90b8915a431ed0126a85fe49"} Jul 07 11:15:45.702805 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.702774 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pr79k" event={"ID":"19149fd0-add1-41dc-bfde-eda95fea0859","Type":"ContainerStarted","Data":"12d7abe0c414df3408647655671f4e57b0800900064066c273befd392c197d71"} Jul 07 11:15:45.711296 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.711267 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" event={"ID":"bc7950df-04e9-423e-9390-0e5f881efe4a","Type":"ContainerStarted","Data":"f8861e9ecc1fcb13a56d55629d96506ab197a8b86a8f2a60bc9874bdc623f282"} Jul 07 11:15:45.730860 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.730789 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" event={"ID":"44b7522ba8d1c0a5a9cbf1723295ee5c","Type":"ContainerStarted","Data":"adb4de71f856ad2f80b491da9f6ee81b1cbd27de0259570fd63eb88b6f9e7641"} Jul 07 11:15:45.740501 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.740401 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfs6d" event={"ID":"0c92ffd4-b8cb-4485-827e-206cf7e95421","Type":"ContainerStarted","Data":"309f13fbd6dbd9d800f32782f5bd207ca3755247e43eb45bc59b04c2a5d93af8"} Jul 07 11:15:45.746243 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.746186 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9qxtj" event={"ID":"2bd8b32a-fafe-400a-8f05-34f364bf41dd","Type":"ContainerStarted","Data":"042ab05dd77175db42614c0d378f3c4a0edf657928e71596c463d8fa66eb1326"} Jul 07 11:15:45.758342 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.758265 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qw6cr" event={"ID":"c29cf0f1-4400-493b-8746-91f03ba47db6","Type":"ContainerStarted","Data":"378fca676801e757ae016a2c24a7471058675e3726d84312a491c83b6a04405e"} Jul 07 11:15:45.772839 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.772804 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-166.ec2.internal" event={"ID":"b14f96639c7f7d472ab32a017c1adf2c","Type":"ContainerStarted","Data":"5bcf19c9670391f96130b4aea0c08de35849a764ef9ea4bcee9aaf01bdbf193c"} Jul 07 11:15:45.798768 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:45.798739 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:46.080847 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:46.080760 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:46.080847 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:46.080810 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:46.081091 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:46.080940 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:46.081091 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:46.081011 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret podName:45bf26b3-aacb-4588-bf31-04a417518c64 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:48.080991426 +0000 UTC m=+5.668857922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret") pod "global-pull-secret-syncer-5zc94" (UID: "45bf26b3-aacb-4588-bf31-04a417518c64") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:46.081457 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:46.081437 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:46.081557 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:46.081497 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs podName:5da61a2d-4113-4869-99bf-f0d24e5edf97 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:48.081480861 +0000 UTC m=+5.669347334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs") pod "network-metrics-daemon-9rwbg" (UID: "5da61a2d-4113-4869-99bf-f0d24e5edf97") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:46.181180 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:46.181140 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cjbm\" (UniqueName: \"kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm\") pod \"network-check-target-kzrqj\" (UID: \"617abf3b-a72a-4600-8764-60aab5668bcd\") " pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:46.181460 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:46.181378 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:15:46.181460 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:46.181415 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:15:46.181460 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:46.181428 2585 projected.go:196] Error preparing data for projected volume kube-api-access-5cjbm for pod openshift-network-diagnostics/network-check-target-kzrqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:46.181673 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:46.181488 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm podName:617abf3b-a72a-4600-8764-60aab5668bcd nodeName:}" failed. No retries permitted until 2026-07-07 11:15:48.18146986 +0000 UTC m=+5.769336333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cjbm" (UniqueName: "kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm") pod "network-check-target-kzrqj" (UID: "617abf3b-a72a-4600-8764-60aab5668bcd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:46.508749 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:46.508628 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:10:45 +0000 UTC" deadline="2028-04-04 14:32:00.4442387 +0000 UTC" Jul 07 11:15:46.509241 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:46.508696 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15291h16m13.935547324s" Jul 07 11:15:46.670362 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:46.669847 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:46.670362 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:46.669981 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:15:46.670362 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:46.670119 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:46.670362 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:46.670256 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:15:46.794802 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:46.794701 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9dcwb" event={"ID":"a648f49b-76f9-44b9-9165-d96f40f7f836","Type":"ContainerStarted","Data":"ae8d5f0aa4ae6c9097c786587a0b1a11ad851dbdc9c6c809ccca8b2bca9c8e19"} Jul 07 11:15:47.182362 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:47.182231 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:47.672646 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:47.672019 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:47.672646 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:47.672155 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:15:48.100241 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:48.099140 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:48.100241 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:48.099191 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:48.100241 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:48.099443 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:48.100241 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:48.099604 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret podName:45bf26b3-aacb-4588-bf31-04a417518c64 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:52.099492781 +0000 UTC m=+9.687359267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret") pod "global-pull-secret-syncer-5zc94" (UID: "45bf26b3-aacb-4588-bf31-04a417518c64") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:48.100241 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:48.100024 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:48.100241 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:48.100106 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs podName:5da61a2d-4113-4869-99bf-f0d24e5edf97 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:52.100089095 +0000 UTC m=+9.687955569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs") pod "network-metrics-daemon-9rwbg" (UID: "5da61a2d-4113-4869-99bf-f0d24e5edf97") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:48.199715 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:48.199673 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cjbm\" (UniqueName: \"kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm\") pod \"network-check-target-kzrqj\" (UID: \"617abf3b-a72a-4600-8764-60aab5668bcd\") " pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:48.199921 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:48.199883 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:15:48.199921 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:48.199909 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:15:48.199921 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:48.199922 2585 projected.go:196] Error preparing data for projected volume kube-api-access-5cjbm for pod openshift-network-diagnostics/network-check-target-kzrqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:48.200105 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:48.199982 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm podName:617abf3b-a72a-4600-8764-60aab5668bcd nodeName:}" failed. No retries permitted until 2026-07-07 11:15:52.199963557 +0000 UTC m=+9.787830033 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cjbm" (UniqueName: "kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm") pod "network-check-target-kzrqj" (UID: "617abf3b-a72a-4600-8764-60aab5668bcd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:48.669442 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:48.669051 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:48.669442 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:48.669200 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:15:48.669442 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:48.669289 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:48.669442 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:48.669444 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:15:49.670424 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:49.670383 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:49.670923 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:49.670541 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:15:50.669813 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:50.669776 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:50.669813 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:50.669797 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:50.670013 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:50.669938 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:15:50.670127 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:50.670090 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:15:51.669787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:51.669752 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:51.670301 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:51.669893 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:15:52.142476 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:52.142356 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:52.142476 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:52.142435 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:52.142698 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:52.142542 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:52.142698 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:52.142585 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:52.142698 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:52.142627 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs podName:5da61a2d-4113-4869-99bf-f0d24e5edf97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:00.142603567 +0000 UTC m=+17.730470043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs") pod "network-metrics-daemon-9rwbg" (UID: "5da61a2d-4113-4869-99bf-f0d24e5edf97") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:52.142698 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:52.142648 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret podName:45bf26b3-aacb-4588-bf31-04a417518c64 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:00.142638238 +0000 UTC m=+17.730504712 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret") pod "global-pull-secret-syncer-5zc94" (UID: "45bf26b3-aacb-4588-bf31-04a417518c64") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:52.243311 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:52.243269 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cjbm\" (UniqueName: \"kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm\") pod \"network-check-target-kzrqj\" (UID: \"617abf3b-a72a-4600-8764-60aab5668bcd\") " pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:52.243541 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:52.243482 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:15:52.243541 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:52.243502 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:15:52.243541 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:52.243516 2585 projected.go:196] Error preparing data for projected volume kube-api-access-5cjbm for pod openshift-network-diagnostics/network-check-target-kzrqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:52.243725 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:52.243581 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm podName:617abf3b-a72a-4600-8764-60aab5668bcd nodeName:}" failed. No retries permitted until 2026-07-07 11:16:00.243560249 +0000 UTC m=+17.831426738 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cjbm" (UniqueName: "kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm") pod "network-check-target-kzrqj" (UID: "617abf3b-a72a-4600-8764-60aab5668bcd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:52.670009 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:52.669976 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:52.670009 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:52.670013 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:52.670539 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:52.670110 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:15:52.670539 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:52.670333 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:15:53.674329 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:53.672152 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:53.674329 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:53.672302 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:15:54.668992 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:54.668969 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:54.669089 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:54.668980 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:54.669089 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:54.669071 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:15:54.669198 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:54.669178 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:15:55.669209 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:55.669175 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:55.669683 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:55.669288 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:15:56.669760 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:56.669727 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:56.670183 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:56.669770 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:56.670183 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:56.669872 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:15:56.670183 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:56.669996 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:15:57.670050 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:57.670014 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:57.670511 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:57.670147 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:15:58.669929 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:58.669886 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:15:58.669929 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:58.669925 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:15:58.670525 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:58.670024 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:15:58.670525 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:58.670158 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:15:59.669607 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:15:59.669573 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:15:59.669794 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:15:59.669702 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:00.206961 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:00.206924 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:00.207481 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:00.206970 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:00.207481 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:00.207102 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:00.207481 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:00.207105 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:00.207481 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:00.207174 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret podName:45bf26b3-aacb-4588-bf31-04a417518c64 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:16.207153995 +0000 UTC m=+33.795020489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret") pod "global-pull-secret-syncer-5zc94" (UID: "45bf26b3-aacb-4588-bf31-04a417518c64") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:00.207481 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:00.207200 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs podName:5da61a2d-4113-4869-99bf-f0d24e5edf97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:16.207186692 +0000 UTC m=+33.795053166 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs") pod "network-metrics-daemon-9rwbg" (UID: "5da61a2d-4113-4869-99bf-f0d24e5edf97") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:00.307974 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:00.307941 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cjbm\" (UniqueName: \"kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm\") pod \"network-check-target-kzrqj\" (UID: \"617abf3b-a72a-4600-8764-60aab5668bcd\") " pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:00.308151 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:00.308095 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:16:00.308151 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:00.308113 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:16:00.308151 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:00.308122 2585 projected.go:196] Error preparing data for projected volume kube-api-access-5cjbm for pod openshift-network-diagnostics/network-check-target-kzrqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:00.308293 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:00.308181 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm podName:617abf3b-a72a-4600-8764-60aab5668bcd nodeName:}" failed. No retries permitted until 2026-07-07 11:16:16.308163735 +0000 UTC m=+33.896030204 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cjbm" (UniqueName: "kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm") pod "network-check-target-kzrqj" (UID: "617abf3b-a72a-4600-8764-60aab5668bcd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:00.669468 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:00.669431 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:00.669678 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:00.669431 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:00.669678 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:00.669575 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:00.669794 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:00.669679 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:01.669220 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:01.669184 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:01.669653 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:01.669323 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:02.669188 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:02.669157 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:02.669395 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:02.669155 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:02.669395 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:02.669298 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:02.669395 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:02.669354 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:03.670637 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:03.670601 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:03.671057 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:03.670722 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:04.669264 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:04.669231 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:04.669453 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:04.669273 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:04.669453 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:04.669370 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:04.669553 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:04.669462 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:05.670070 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:05.669823 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:05.670512 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:05.670167 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:05.838863 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:05.838834 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" event={"ID":"9609fab6-8573-4b1d-8a7b-add43d375ecc","Type":"ContainerStarted","Data":"e7fcf13d14351dc9890dcf4850d65a44278129e1b27b9eb7f01d5258fc7d729f"} Jul 07 11:16:05.838996 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:05.838912 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" event={"ID":"9609fab6-8573-4b1d-8a7b-add43d375ecc","Type":"ContainerStarted","Data":"2d656735dd830899a339f76ff47cf52a06ff597109acd1127d61b4b4de158b97"} Jul 07 11:16:05.840216 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:05.840182 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pr79k" event={"ID":"19149fd0-add1-41dc-bfde-eda95fea0859","Type":"ContainerStarted","Data":"5cb2daa680030cc33f5be6ba10ab776771e7ef39e76f709f4549a4a7f5f66bfc"} Jul 07 11:16:05.841868 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:05.841733 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9dcwb" event={"ID":"a648f49b-76f9-44b9-9165-d96f40f7f836","Type":"ContainerStarted","Data":"03d676e7a7b83b36f3f9ecaef9b271267ddeca722b378eef11298747933e4608"} Jul 07 11:16:05.843336 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:05.843309 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-166.ec2.internal" event={"ID":"b14f96639c7f7d472ab32a017c1adf2c","Type":"ContainerStarted","Data":"028e21273ea5a2eb13cfce6711899a6f04b9c1cf9c1c1979ea0424ad4dc70a62"} Jul 07 11:16:05.868999 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:05.868951 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pr79k" podStartSLOduration=1.391566882 podStartE2EDuration="21.868934481s" totalImagesPullingTime="20.477367599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:05.868164344 +0000 UTC m=+23.456030835" watchObservedRunningTime="2026-07-07 11:16:05.868934481 +0000 UTC m=+23.456800972" Jul 07 11:16:05.891253 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:05.891198 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-166.ec2.internal" podStartSLOduration=20.891176509 podStartE2EDuration="20.891176509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:05.890950616 +0000 UTC m=+23.478817108" watchObservedRunningTime="2026-07-07 11:16:05.891176509 +0000 UTC m=+23.479043000" Jul 07 11:16:05.915020 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:05.914976 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9dcwb" podStartSLOduration=2.129176411 podStartE2EDuration="21.91496109s" totalImagesPullingTime="19.785784679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:05.91456172 +0000 UTC m=+23.502428209" watchObservedRunningTime="2026-07-07 11:16:05.91496109 +0000 UTC m=+23.502827580" Jul 07 11:16:06.669466 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.669268 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:06.669670 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.669325 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:06.669670 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:06.669566 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:06.669670 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:06.669625 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:06.846475 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.846443 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9qxtj" event={"ID":"2bd8b32a-fafe-400a-8f05-34f364bf41dd","Type":"ContainerStarted","Data":"5a64466765aef0763d13e6060345de2569da7fbe50a2486a64a05d923ae6b721"} Jul 07 11:16:06.847905 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.847871 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qw6cr" event={"ID":"c29cf0f1-4400-493b-8746-91f03ba47db6","Type":"ContainerStarted","Data":"5518645a003943c3665177108b01e01f3103c6a6fcae4220121bd0f5de7e4b24"} Jul 07 11:16:06.850470 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.850450 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:16:06.850771 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.850753 2585 generic.go:356] "Generic (PLEG): container finished" podID="9609fab6-8573-4b1d-8a7b-add43d375ecc" containerID="e7fcf13d14351dc9890dcf4850d65a44278129e1b27b9eb7f01d5258fc7d729f" exitCode=1 Jul 07 11:16:06.850858 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.850829 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" event={"ID":"9609fab6-8573-4b1d-8a7b-add43d375ecc","Type":"ContainerDied","Data":"e7fcf13d14351dc9890dcf4850d65a44278129e1b27b9eb7f01d5258fc7d729f"} Jul 07 11:16:06.850921 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.850907 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" event={"ID":"9609fab6-8573-4b1d-8a7b-add43d375ecc","Type":"ContainerStarted","Data":"e17190c6b695aaa358053379337d3ebae740ad25f470ef57bf5bbaae338442fc"} Jul 07 11:16:06.850971 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.850959 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" event={"ID":"9609fab6-8573-4b1d-8a7b-add43d375ecc","Type":"ContainerStarted","Data":"4b41591cd4d686b25aad500ffee697fe8fc498f6464d96edb416c69dd7344329"} Jul 07 11:16:06.851023 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.851009 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" event={"ID":"9609fab6-8573-4b1d-8a7b-add43d375ecc","Type":"ContainerStarted","Data":"f8f9e3f7b0f7d92f67417014b4431f4607c1ad5cde176f7f832515de26e37466"} Jul 07 11:16:06.851076 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.851057 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" event={"ID":"9609fab6-8573-4b1d-8a7b-add43d375ecc","Type":"ContainerStarted","Data":"3e478dd71a876ec670d68d6d2116a4d9cb7b4af82b3a76cb098c163d647f9f93"} Jul 07 11:16:06.852009 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.851985 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7bn6j" event={"ID":"ef118293-20df-4a0e-8b0f-9204c6096441","Type":"ContainerStarted","Data":"ffa4552d4aa9fd730ad9ca3bad971d55dc3750bc4ad8a2371256889a01101dbf"} Jul 07 11:16:06.853241 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.853219 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kz98j" event={"ID":"3e2bee60-3e0b-403d-80a0-86946959866d","Type":"ContainerStarted","Data":"90eb9f93b52e094ae632633f5aa465444af60e1cb01aeac8d0ea4bc9d515225b"} Jul 07 11:16:06.854575 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.854555 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" event={"ID":"bc7950df-04e9-423e-9390-0e5f881efe4a","Type":"ContainerStarted","Data":"eac317f7ed446c6218fc9396eb21372ad7df1102c1cc5170fdeecd2e0f908ea0"} Jul 07 11:16:06.855801 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.855781 2585 generic.go:356] "Generic (PLEG): container finished" podID="44b7522ba8d1c0a5a9cbf1723295ee5c" containerID="b140c7e08e7e3b08341c3ae3a8ef798266a91e348adff464262a39de6e8bf1fc" exitCode=0 Jul 07 11:16:06.855882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.855854 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" event={"ID":"44b7522ba8d1c0a5a9cbf1723295ee5c","Type":"ContainerDied","Data":"b140c7e08e7e3b08341c3ae3a8ef798266a91e348adff464262a39de6e8bf1fc"} Jul 07 11:16:06.857388 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.857369 2585 generic.go:356] "Generic (PLEG): container finished" podID="0c92ffd4-b8cb-4485-827e-206cf7e95421" containerID="0f174c04d82c4543e666c838d2f9f5d9aac1266e2414bd73df4c1176d60ba415" exitCode=0 Jul 07 11:16:06.857501 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.857473 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfs6d" event={"ID":"0c92ffd4-b8cb-4485-827e-206cf7e95421","Type":"ContainerDied","Data":"0f174c04d82c4543e666c838d2f9f5d9aac1266e2414bd73df4c1176d60ba415"} Jul 07 11:16:06.862282 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.862246 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9qxtj" podStartSLOduration=3.052381997 podStartE2EDuration="22.862236157s" totalImagesPullingTime="19.80985416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:06.862180091 +0000 UTC m=+24.450046578" watchObservedRunningTime="2026-07-07 11:16:06.862236157 +0000 UTC m=+24.450102647" Jul 07 11:16:06.879683 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.879643 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qw6cr" podStartSLOduration=2.398224629 podStartE2EDuration="22.879625709s" totalImagesPullingTime="20.48140108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:06.87849705 +0000 UTC m=+24.466363542" watchObservedRunningTime="2026-07-07 11:16:06.879625709 +0000 UTC m=+24.467492201" Jul 07 11:16:06.895455 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.895410 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kz98j" podStartSLOduration=2.467065395 podStartE2EDuration="22.895397803s" totalImagesPullingTime="20.428332408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:06.894863432 +0000 UTC m=+24.482729922" watchObservedRunningTime="2026-07-07 11:16:06.895397803 +0000 UTC m=+24.483264295" Jul 07 11:16:06.914441 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.914402 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7bn6j" podStartSLOduration=2.4850879040000002 podStartE2EDuration="22.914387188s" totalImagesPullingTime="20.429299284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:06.913995491 +0000 UTC m=+24.501861983" watchObservedRunningTime="2026-07-07 11:16:06.914387188 +0000 UTC m=+24.502253672" Jul 07 11:16:06.958380 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.958010 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:16:06.960303 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:06.960131 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:16:07.670136 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:07.670100 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:07.670303 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:07.670230 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:07.706495 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:07.706451 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 11:16:07.860684 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:07.860610 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" event={"ID":"bc7950df-04e9-423e-9390-0e5f881efe4a","Type":"ContainerStarted","Data":"a1858940c2ea8add52868ec79c1671621f0ae6373e3f09f906865e79f887b661"} Jul 07 11:16:07.862538 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:07.862508 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" event={"ID":"44b7522ba8d1c0a5a9cbf1723295ee5c","Type":"ContainerStarted","Data":"785b67fcf2804ee7e927888294a978ef84083359881012c42956b9ebe037b09b"} Jul 07 11:16:07.862711 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:07.862695 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:16:07.863201 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:07.863183 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qw6cr" Jul 07 11:16:07.881046 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:07.880999 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-166.ec2.internal" podStartSLOduration=22.880983554 podStartE2EDuration="22.880983554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:07.880913827 +0000 UTC m=+25.468780318" watchObservedRunningTime="2026-07-07 11:16:07.880983554 +0000 UTC m=+25.468850044" Jul 07 11:16:08.537110 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:08.536760 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T11:16:07.706487471Z","UUID":"5eb08361-b550-4535-9811-cc1c1a253cfb","Handler":null,"Name":"","Endpoint":""} Jul 07 11:16:08.539333 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:08.539276 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 11:16:08.539333 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:08.539320 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 11:16:08.669553 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:08.669513 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:08.669739 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:08.669516 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:08.669739 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:08.669627 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:08.669840 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:08.669741 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:08.868878 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:08.868702 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:16:08.869304 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:08.869267 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" event={"ID":"9609fab6-8573-4b1d-8a7b-add43d375ecc","Type":"ContainerStarted","Data":"2b66eedf1b46c02f345033c348d75e247f23885c9d64a59df6e4e821da555b05"} Jul 07 11:16:09.669569 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:09.669521 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:09.669758 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:09.669638 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:09.873677 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:09.873641 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" event={"ID":"bc7950df-04e9-423e-9390-0e5f881efe4a","Type":"ContainerStarted","Data":"1860544a6d23157fdd9b25d2aa38c3d0c54476ad8eb37189929fc6b01a4b906b"} Jul 07 11:16:10.669748 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:10.669705 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:10.669748 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:10.669736 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:10.669967 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:10.669844 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:10.670008 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:10.669961 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:11.669579 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:11.669543 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:11.670086 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:11.669660 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:11.881803 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:11.881672 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:16:11.882131 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:11.882108 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" event={"ID":"9609fab6-8573-4b1d-8a7b-add43d375ecc","Type":"ContainerStarted","Data":"3b98d37daff795e1f06ece499eee92f4e01438281040f4f8767b6eafce81a1ac"} Jul 07 11:16:11.884997 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:11.883048 2585 scope.go:117] "RemoveContainer" containerID="e7fcf13d14351dc9890dcf4850d65a44278129e1b27b9eb7f01d5258fc7d729f" Jul 07 11:16:11.884997 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:11.883944 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:16:11.884997 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:11.884065 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:16:11.900263 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:11.900239 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:16:11.916669 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:11.916626 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k8mfc" podStartSLOduration=4.90200066 podStartE2EDuration="27.91661036s" totalImagesPullingTime="23.0146097s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:09.907854278 +0000 UTC m=+27.495720768" watchObservedRunningTime="2026-07-07 11:16:11.91661036 +0000 UTC m=+29.504476849" Jul 07 11:16:12.669479 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:12.669438 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:12.669760 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:12.669453 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:12.669760 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:12.669582 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:12.669760 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:12.669626 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:12.887944 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:12.887915 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:16:12.888321 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:12.888297 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" event={"ID":"9609fab6-8573-4b1d-8a7b-add43d375ecc","Type":"ContainerStarted","Data":"1cbb7d33a9a8b587ee79d145fa8efe73c1c5d1849d165cf7e154cb3720e2f579"} Jul 07 11:16:12.888615 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:12.888590 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:16:12.890132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:12.890101 2585 generic.go:356] "Generic (PLEG): container finished" podID="0c92ffd4-b8cb-4485-827e-206cf7e95421" containerID="40e00d300edc76b1989950e3d9aae7f3e7149b3cffef639d0e690a413d830730" exitCode=0 Jul 07 11:16:12.890218 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:12.890134 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfs6d" event={"ID":"0c92ffd4-b8cb-4485-827e-206cf7e95421","Type":"ContainerDied","Data":"40e00d300edc76b1989950e3d9aae7f3e7149b3cffef639d0e690a413d830730"} Jul 07 11:16:12.904197 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:12.904178 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:16:12.925891 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:12.925812 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" podStartSLOduration=8.905130333 podStartE2EDuration="28.925797863s" totalImagesPullingTime="20.02066753s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:12.92550536 +0000 UTC m=+30.513371849" watchObservedRunningTime="2026-07-07 11:16:12.925797863 +0000 UTC m=+30.513664353" Jul 07 11:16:13.670191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:13.670153 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:13.670640 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:13.670239 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:14.669643 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:14.669607 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:14.669643 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:14.669640 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:14.669981 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:14.669754 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:14.669981 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:14.669843 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:14.897268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:14.897236 2585 generic.go:356] "Generic (PLEG): container finished" podID="0c92ffd4-b8cb-4485-827e-206cf7e95421" containerID="c5533f365c22a33a612008e342c8512863beffa3d0bb974eda5d56f95fbf8b9d" exitCode=0 Jul 07 11:16:14.897635 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:14.897324 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfs6d" event={"ID":"0c92ffd4-b8cb-4485-827e-206cf7e95421","Type":"ContainerDied","Data":"c5533f365c22a33a612008e342c8512863beffa3d0bb974eda5d56f95fbf8b9d"} Jul 07 11:16:15.669706 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:15.669674 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:15.669866 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:15.669781 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:16.223466 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:16.223435 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:16.223749 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:16.223470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:16.223749 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:16.223565 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:16.223749 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:16.223597 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:16.223749 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:16.223623 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret podName:45bf26b3-aacb-4588-bf31-04a417518c64 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:48.223610036 +0000 UTC m=+65.811476505 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret") pod "global-pull-secret-syncer-5zc94" (UID: "45bf26b3-aacb-4588-bf31-04a417518c64") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:16.223749 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:16.223655 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs podName:5da61a2d-4113-4869-99bf-f0d24e5edf97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:48.223638223 +0000 UTC m=+65.811504710 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs") pod "network-metrics-daemon-9rwbg" (UID: "5da61a2d-4113-4869-99bf-f0d24e5edf97") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:16.323883 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:16.323786 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cjbm\" (UniqueName: \"kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm\") pod \"network-check-target-kzrqj\" (UID: \"617abf3b-a72a-4600-8764-60aab5668bcd\") " pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:16.324024 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:16.323945 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:16:16.324024 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:16.323966 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:16:16.324024 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:16.323976 2585 projected.go:196] Error preparing data for projected volume kube-api-access-5cjbm for pod openshift-network-diagnostics/network-check-target-kzrqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:16.324150 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:16.324025 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm podName:617abf3b-a72a-4600-8764-60aab5668bcd nodeName:}" failed. No retries permitted until 2026-07-07 11:16:48.324011776 +0000 UTC m=+65.911878244 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5cjbm" (UniqueName: "kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm") pod "network-check-target-kzrqj" (UID: "617abf3b-a72a-4600-8764-60aab5668bcd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:16.669022 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:16.668988 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:16.669198 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:16.668988 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:16.669198 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:16.669093 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:16.669198 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:16.669161 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:16.903947 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:16.903908 2585 generic.go:356] "Generic (PLEG): container finished" podID="0c92ffd4-b8cb-4485-827e-206cf7e95421" containerID="620e7b514b1e8c2ea2a987e18b7fbc1f2fcc1bc7c6241863c03bf18798d390ca" exitCode=0 Jul 07 11:16:16.904118 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:16.903955 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfs6d" event={"ID":"0c92ffd4-b8cb-4485-827e-206cf7e95421","Type":"ContainerDied","Data":"620e7b514b1e8c2ea2a987e18b7fbc1f2fcc1bc7c6241863c03bf18798d390ca"} Jul 07 11:16:17.669221 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:17.669191 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:17.669709 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:17.669318 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:18.669662 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:18.669625 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:18.670121 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:18.669743 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:18.670121 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:18.669803 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:18.670121 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:18.669934 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:19.669241 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:19.669206 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:19.669446 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:19.669329 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:20.669285 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:20.669249 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:20.669704 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:20.669292 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:20.669704 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:20.669414 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:20.669704 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:20.669531 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:21.670093 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:21.669917 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:21.670485 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:21.670177 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:21.918833 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:21.918800 2585 generic.go:356] "Generic (PLEG): container finished" podID="0c92ffd4-b8cb-4485-827e-206cf7e95421" containerID="b04d896823b5a0a9169e1473922face449622041859d796cba3a3e12ff10bd0b" exitCode=0 Jul 07 11:16:21.919018 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:21.918854 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfs6d" event={"ID":"0c92ffd4-b8cb-4485-827e-206cf7e95421","Type":"ContainerDied","Data":"b04d896823b5a0a9169e1473922face449622041859d796cba3a3e12ff10bd0b"} Jul 07 11:16:22.670009 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:22.669979 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:22.670168 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:22.670072 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:22.670168 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:22.670133 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:22.670471 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:22.670226 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:22.924318 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:22.924224 2585 generic.go:356] "Generic (PLEG): container finished" podID="0c92ffd4-b8cb-4485-827e-206cf7e95421" containerID="df981d4129a125282d443086622a88326acde8ce721c4febf5925ec056a21e58" exitCode=0 Jul 07 11:16:22.924318 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:22.924264 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfs6d" event={"ID":"0c92ffd4-b8cb-4485-827e-206cf7e95421","Type":"ContainerDied","Data":"df981d4129a125282d443086622a88326acde8ce721c4febf5925ec056a21e58"} Jul 07 11:16:23.670952 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:23.670910 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:23.671326 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:23.671033 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:23.930193 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:23.930107 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rfs6d" event={"ID":"0c92ffd4-b8cb-4485-827e-206cf7e95421","Type":"ContainerStarted","Data":"82a3be49611aa3ba40486048bf6fcbf64597716679fec4a4c282447505388d12"} Jul 07 11:16:23.958322 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:23.958274 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rfs6d" podStartSLOduration=12.493587021 podStartE2EDuration="39.958259897s" totalImagesPullingTime="27.464672876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:23.957769674 +0000 UTC m=+41.545636165" watchObservedRunningTime="2026-07-07 11:16:23.958259897 +0000 UTC m=+41.546126387" Jul 07 11:16:24.669196 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:24.669157 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:24.669418 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:24.669157 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:24.669418 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:24.669285 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:24.669418 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:24.669385 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:25.669884 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:25.669844 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:25.670247 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:25.669993 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:26.669133 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:26.669101 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:26.669277 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:26.669102 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:26.669277 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:26.669205 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:26.669407 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:26.669279 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:27.670000 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:27.669967 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:27.670399 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:27.670070 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:28.669147 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:28.669115 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:28.669147 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:28.669150 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:28.669374 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:28.669234 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:28.669374 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:28.669365 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:29.670006 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:29.669975 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:29.670533 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:29.670075 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:30.031322 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:30.031292 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9rwbg"] Jul 07 11:16:30.031476 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:30.031407 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:30.031519 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:30.031494 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:30.033645 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:30.033618 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5zc94"] Jul 07 11:16:30.033782 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:30.033700 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:30.033848 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:30.033783 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:30.034106 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:30.034082 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kzrqj"] Jul 07 11:16:30.034191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:30.034173 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:30.034289 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:30.034267 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:31.669567 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:31.669533 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:31.670118 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:31.669533 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:31.670118 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:31.669641 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5zc94" podUID="45bf26b3-aacb-4588-bf31-04a417518c64" Jul 07 11:16:31.670118 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:31.669730 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kzrqj" podUID="617abf3b-a72a-4600-8764-60aab5668bcd" Jul 07 11:16:31.670118 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:31.669550 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:31.670118 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:31.669818 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rwbg" podUID="5da61a2d-4113-4869-99bf-f0d24e5edf97" Jul 07 11:16:33.216816 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.216642 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-166.ec2.internal" event="NodeReady" Jul 07 11:16:33.217159 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.216910 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 11:16:33.255468 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.255435 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bb544648c-4lkkg"] Jul 07 11:16:33.267468 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.267365 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.271323 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.271298 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 11:16:33.272490 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.272472 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 11:16:33.272586 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.272479 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rbld5\"" Jul 07 11:16:33.275007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.274985 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 11:16:33.278322 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.278299 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 11:16:33.278789 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.278766 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bb544648c-4lkkg"] Jul 07 11:16:33.282282 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.282224 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8cpdb"] Jul 07 11:16:33.302164 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.302138 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l6xpr"] Jul 07 11:16:33.302336 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.302291 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.305849 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.305683 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgd79\"" Jul 07 11:16:33.305849 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.305686 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 11:16:33.305849 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.305774 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 11:16:33.315236 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.315220 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l6xpr"] Jul 07 11:16:33.315307 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.315245 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8cpdb"] Jul 07 11:16:33.315365 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.315318 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l6xpr" Jul 07 11:16:33.318997 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.318974 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 11:16:33.319083 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.319043 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hr2xm\"" Jul 07 11:16:33.319083 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.319058 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 11:16:33.319297 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.319282 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 11:16:33.350180 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350158 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-config-volume\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.350280 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350199 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp8qr\" (UniqueName: \"kubernetes.io/projected/44a96807-d2da-429a-9ee8-27a92649ac56-kube-api-access-mp8qr\") pod \"ingress-canary-l6xpr\" (UID: \"44a96807-d2da-429a-9ee8-27a92649ac56\") " pod="openshift-ingress-canary/ingress-canary-l6xpr" Jul 07 11:16:33.350280 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350225 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zhfq\" (UniqueName: \"kubernetes.io/projected/3177f6fc-71ff-4a37-939a-9ffe0611172b-kube-api-access-8zhfq\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.350280 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350261 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-metrics-tls\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.350407 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350316 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3177f6fc-71ff-4a37-939a-9ffe0611172b-registry-tls\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.350407 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350333 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3177f6fc-71ff-4a37-939a-9ffe0611172b-trusted-ca\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.350407 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350366 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3177f6fc-71ff-4a37-939a-9ffe0611172b-image-registry-private-configuration\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.350407 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350383 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3177f6fc-71ff-4a37-939a-9ffe0611172b-bound-sa-token\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.350524 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350416 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44a96807-d2da-429a-9ee8-27a92649ac56-cert\") pod \"ingress-canary-l6xpr\" (UID: \"44a96807-d2da-429a-9ee8-27a92649ac56\") " pod="openshift-ingress-canary/ingress-canary-l6xpr" Jul 07 11:16:33.350524 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350437 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3177f6fc-71ff-4a37-939a-9ffe0611172b-registry-certificates\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.350524 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350475 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkwbz\" (UniqueName: \"kubernetes.io/projected/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-kube-api-access-kkwbz\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.350524 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350501 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3177f6fc-71ff-4a37-939a-9ffe0611172b-installation-pull-secrets\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.350649 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350550 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-tmp-dir\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.350649 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.350573 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3177f6fc-71ff-4a37-939a-9ffe0611172b-ca-trust-extracted\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.394866 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.394837 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g8vcl"] Jul 07 11:16:33.412540 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.412515 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw"] Jul 07 11:16:33.412662 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.412647 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.418833 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.418813 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 11:16:33.418935 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.418830 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 11:16:33.419265 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.419238 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 11:16:33.419508 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.419491 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 11:16:33.419585 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.419528 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qdxb4\"" Jul 07 11:16:33.431236 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.431214 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g8vcl"] Jul 07 11:16:33.431334 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.431316 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw" Jul 07 11:16:33.434882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.434867 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 11:16:33.434988 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.434929 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-whr92\"" Jul 07 11:16:33.438277 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.438258 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw"] Jul 07 11:16:33.451386 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451367 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkwbz\" (UniqueName: \"kubernetes.io/projected/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-kube-api-access-kkwbz\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.451482 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451398 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3177f6fc-71ff-4a37-939a-9ffe0611172b-installation-pull-secrets\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.451482 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451429 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-tmp-dir\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.451482 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451449 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3177f6fc-71ff-4a37-939a-9ffe0611172b-ca-trust-extracted\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.451482 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451467 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-config-volume\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.451725 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451486 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.451725 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451514 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp8qr\" (UniqueName: \"kubernetes.io/projected/44a96807-d2da-429a-9ee8-27a92649ac56-kube-api-access-mp8qr\") pod \"ingress-canary-l6xpr\" (UID: \"44a96807-d2da-429a-9ee8-27a92649ac56\") " pod="openshift-ingress-canary/ingress-canary-l6xpr" Jul 07 11:16:33.451725 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451538 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.451725 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451611 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zhfq\" (UniqueName: \"kubernetes.io/projected/3177f6fc-71ff-4a37-939a-9ffe0611172b-kube-api-access-8zhfq\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.451725 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451645 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-metrics-tls\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.451725 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451673 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-crio-socket\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.451725 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451698 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3177f6fc-71ff-4a37-939a-9ffe0611172b-registry-tls\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.451725 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451720 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3177f6fc-71ff-4a37-939a-9ffe0611172b-trusted-ca\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.452173 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451741 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-data-volume\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.452173 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451768 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3177f6fc-71ff-4a37-939a-9ffe0611172b-image-registry-private-configuration\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.452173 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451807 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-tmp-dir\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.452173 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451807 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3177f6fc-71ff-4a37-939a-9ffe0611172b-bound-sa-token\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.452173 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451847 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb47r\" (UniqueName: \"kubernetes.io/projected/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-kube-api-access-tb47r\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.452173 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451876 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3177f6fc-71ff-4a37-939a-9ffe0611172b-ca-trust-extracted\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.452173 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451890 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44a96807-d2da-429a-9ee8-27a92649ac56-cert\") pod \"ingress-canary-l6xpr\" (UID: \"44a96807-d2da-429a-9ee8-27a92649ac56\") " pod="openshift-ingress-canary/ingress-canary-l6xpr" Jul 07 11:16:33.452173 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.451945 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e001e573-d466-497b-8408-9f0f7d098d57-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-9bwkw\" (UID: \"e001e573-d466-497b-8408-9f0f7d098d57\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw" Jul 07 11:16:33.452173 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.452029 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3177f6fc-71ff-4a37-939a-9ffe0611172b-registry-certificates\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.452626 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.452289 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-config-volume\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.452793 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.452772 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3177f6fc-71ff-4a37-939a-9ffe0611172b-registry-certificates\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.453003 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.452980 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3177f6fc-71ff-4a37-939a-9ffe0611172b-trusted-ca\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.455898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.455880 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3177f6fc-71ff-4a37-939a-9ffe0611172b-installation-pull-secrets\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.455898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.455885 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-metrics-tls\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.455898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.455894 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44a96807-d2da-429a-9ee8-27a92649ac56-cert\") pod \"ingress-canary-l6xpr\" (UID: \"44a96807-d2da-429a-9ee8-27a92649ac56\") " pod="openshift-ingress-canary/ingress-canary-l6xpr" Jul 07 11:16:33.456046 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.455941 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3177f6fc-71ff-4a37-939a-9ffe0611172b-registry-tls\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.456046 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.456006 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3177f6fc-71ff-4a37-939a-9ffe0611172b-image-registry-private-configuration\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.463076 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.463058 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkwbz\" (UniqueName: \"kubernetes.io/projected/7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf-kube-api-access-kkwbz\") pod \"dns-default-8cpdb\" (UID: \"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf\") " pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.465911 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.465869 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3177f6fc-71ff-4a37-939a-9ffe0611172b-bound-sa-token\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.466041 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.465951 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zhfq\" (UniqueName: \"kubernetes.io/projected/3177f6fc-71ff-4a37-939a-9ffe0611172b-kube-api-access-8zhfq\") pod \"image-registry-bb544648c-4lkkg\" (UID: \"3177f6fc-71ff-4a37-939a-9ffe0611172b\") " pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.466196 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.466178 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp8qr\" (UniqueName: \"kubernetes.io/projected/44a96807-d2da-429a-9ee8-27a92649ac56-kube-api-access-mp8qr\") pod \"ingress-canary-l6xpr\" (UID: \"44a96807-d2da-429a-9ee8-27a92649ac56\") " pod="openshift-ingress-canary/ingress-canary-l6xpr" Jul 07 11:16:33.553428 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.553302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.553428 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.553395 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.553428 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.553431 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-crio-socket\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.553671 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.553456 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-data-volume\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.553671 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.553488 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb47r\" (UniqueName: \"kubernetes.io/projected/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-kube-api-access-tb47r\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.553671 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.553522 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e001e573-d466-497b-8408-9f0f7d098d57-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-9bwkw\" (UID: \"e001e573-d466-497b-8408-9f0f7d098d57\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw" Jul 07 11:16:33.553671 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.553624 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-crio-socket\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.553860 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.553840 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-data-volume\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.554055 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.554035 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.555738 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.555712 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e001e573-d466-497b-8408-9f0f7d098d57-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-9bwkw\" (UID: \"e001e573-d466-497b-8408-9f0f7d098d57\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw" Jul 07 11:16:33.555858 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.555839 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.566469 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.566450 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb47r\" (UniqueName: \"kubernetes.io/projected/ed8c5e9b-6266-46f1-b394-53fc8ecc1056-kube-api-access-tb47r\") pod \"insights-runtime-extractor-g8vcl\" (UID: \"ed8c5e9b-6266-46f1-b394-53fc8ecc1056\") " pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.578454 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.578434 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.612823 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.612802 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:33.622918 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.622889 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l6xpr" Jul 07 11:16:33.670856 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.670644 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:33.673465 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.671065 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:33.673465 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.671713 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:33.681363 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.681149 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 11:16:33.681546 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.681427 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89jkb\"" Jul 07 11:16:33.681678 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.681664 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 11:16:33.682175 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.681840 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42fbv\"" Jul 07 11:16:33.682373 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.682282 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 11:16:33.683003 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.682808 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 11:16:33.721674 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.721641 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g8vcl" Jul 07 11:16:33.740293 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.740192 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw" Jul 07 11:16:33.792455 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.792423 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-2chlh"] Jul 07 11:16:33.802727 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.802700 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-2chlh" Jul 07 11:16:33.806911 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.806837 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7sj7b\"" Jul 07 11:16:33.807159 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.807143 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 11:16:33.807414 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.807398 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 11:16:33.813490 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.813468 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-2chlh"] Jul 07 11:16:33.840894 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.837843 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l6xpr"] Jul 07 11:16:33.840894 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.838932 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8cpdb"] Jul 07 11:16:33.844871 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:16:33.844675 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a96807_d2da_429a_9ee8_27a92649ac56.slice/crio-7ddd532aa489e34b1182eaefe1d819e2139cf84a1d017acb02f318b52e06e26d WatchSource:0}: Error finding container 7ddd532aa489e34b1182eaefe1d819e2139cf84a1d017acb02f318b52e06e26d: Status 404 returned error can't find the container with id 7ddd532aa489e34b1182eaefe1d819e2139cf84a1d017acb02f318b52e06e26d Jul 07 11:16:33.845408 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:16:33.845305 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dfa4e40_8be9_4dde_91b0_9b5e812b6fbf.slice/crio-1dcd7f5a3d2127244bcd47981eebd5b40c76786affe6fe8a879c8d179221ee49 WatchSource:0}: Error finding container 1dcd7f5a3d2127244bcd47981eebd5b40c76786affe6fe8a879c8d179221ee49: Status 404 returned error can't find the container with id 1dcd7f5a3d2127244bcd47981eebd5b40c76786affe6fe8a879c8d179221ee49 Jul 07 11:16:33.845743 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.845681 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bb544648c-4lkkg"] Jul 07 11:16:33.849534 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:16:33.849507 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3177f6fc_71ff_4a37_939a_9ffe0611172b.slice/crio-cf237b82987d861c3977a36a91f7281908fa263c77615b72bb04edd1b5c6a239 WatchSource:0}: Error finding container cf237b82987d861c3977a36a91f7281908fa263c77615b72bb04edd1b5c6a239: Status 404 returned error can't find the container with id cf237b82987d861c3977a36a91f7281908fa263c77615b72bb04edd1b5c6a239 Jul 07 11:16:33.855050 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.855031 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq8dv\" (UniqueName: \"kubernetes.io/projected/1895fd42-75c7-4daf-8e26-07d5adae8df0-kube-api-access-wq8dv\") pod \"downloads-685bfb5f64-2chlh\" (UID: \"1895fd42-75c7-4daf-8e26-07d5adae8df0\") " pod="openshift-console/downloads-685bfb5f64-2chlh" Jul 07 11:16:33.920910 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.920707 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g8vcl"] Jul 07 11:16:33.921290 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.921270 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw"] Jul 07 11:16:33.922997 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:16:33.922967 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8c5e9b_6266_46f1_b394_53fc8ecc1056.slice/crio-d1f024a5e3b151ef2ea7155ba03a1991b7fa1f5f20b40e2064a548cd32304b71 WatchSource:0}: Error finding container d1f024a5e3b151ef2ea7155ba03a1991b7fa1f5f20b40e2064a548cd32304b71: Status 404 returned error can't find the container with id d1f024a5e3b151ef2ea7155ba03a1991b7fa1f5f20b40e2064a548cd32304b71 Jul 07 11:16:33.923826 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:16:33.923809 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode001e573_d466_497b_8408_9f0f7d098d57.slice/crio-c344ee8d4114cbff416adbf1049131523280fd412ba087d566614e5b3c81aad7 WatchSource:0}: Error finding container c344ee8d4114cbff416adbf1049131523280fd412ba087d566614e5b3c81aad7: Status 404 returned error can't find the container with id c344ee8d4114cbff416adbf1049131523280fd412ba087d566614e5b3c81aad7 Jul 07 11:16:33.954178 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.954150 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8vcl" event={"ID":"ed8c5e9b-6266-46f1-b394-53fc8ecc1056","Type":"ContainerStarted","Data":"d1f024a5e3b151ef2ea7155ba03a1991b7fa1f5f20b40e2064a548cd32304b71"} Jul 07 11:16:33.955324 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.955300 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l6xpr" event={"ID":"44a96807-d2da-429a-9ee8-27a92649ac56","Type":"ContainerStarted","Data":"7ddd532aa489e34b1182eaefe1d819e2139cf84a1d017acb02f318b52e06e26d"} Jul 07 11:16:33.955479 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.955464 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq8dv\" (UniqueName: \"kubernetes.io/projected/1895fd42-75c7-4daf-8e26-07d5adae8df0-kube-api-access-wq8dv\") pod \"downloads-685bfb5f64-2chlh\" (UID: \"1895fd42-75c7-4daf-8e26-07d5adae8df0\") " pod="openshift-console/downloads-685bfb5f64-2chlh" Jul 07 11:16:33.956757 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.956736 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw" event={"ID":"e001e573-d466-497b-8408-9f0f7d098d57","Type":"ContainerStarted","Data":"c344ee8d4114cbff416adbf1049131523280fd412ba087d566614e5b3c81aad7"} Jul 07 11:16:33.958091 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.958071 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bb544648c-4lkkg" event={"ID":"3177f6fc-71ff-4a37-939a-9ffe0611172b","Type":"ContainerStarted","Data":"0271e9f90b7c5a6b8c13a477089ff8364e9c66745a33f58c29bf7ed6ef966388"} Jul 07 11:16:33.958167 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.958109 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bb544648c-4lkkg" event={"ID":"3177f6fc-71ff-4a37-939a-9ffe0611172b","Type":"ContainerStarted","Data":"cf237b82987d861c3977a36a91f7281908fa263c77615b72bb04edd1b5c6a239"} Jul 07 11:16:33.958221 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.958189 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:33.959101 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.959082 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8cpdb" event={"ID":"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf","Type":"ContainerStarted","Data":"1dcd7f5a3d2127244bcd47981eebd5b40c76786affe6fe8a879c8d179221ee49"} Jul 07 11:16:33.972358 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.972331 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq8dv\" (UniqueName: \"kubernetes.io/projected/1895fd42-75c7-4daf-8e26-07d5adae8df0-kube-api-access-wq8dv\") pod \"downloads-685bfb5f64-2chlh\" (UID: \"1895fd42-75c7-4daf-8e26-07d5adae8df0\") " pod="openshift-console/downloads-685bfb5f64-2chlh" Jul 07 11:16:33.994267 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:33.994210 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bb544648c-4lkkg" podStartSLOduration=5.994194352 podStartE2EDuration="5.994194352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:33.992430796 +0000 UTC m=+51.580297288" watchObservedRunningTime="2026-07-07 11:16:33.994194352 +0000 UTC m=+51.582060842" Jul 07 11:16:34.112285 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:34.112194 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-2chlh" Jul 07 11:16:34.261065 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:34.261030 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-2chlh"] Jul 07 11:16:34.265587 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:16:34.265529 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1895fd42_75c7_4daf_8e26_07d5adae8df0.slice/crio-14348f028fe581d89f5a481c0b1bd8e12171ff4150708c1f2cffeec369526271 WatchSource:0}: Error finding container 14348f028fe581d89f5a481c0b1bd8e12171ff4150708c1f2cffeec369526271: Status 404 returned error can't find the container with id 14348f028fe581d89f5a481c0b1bd8e12171ff4150708c1f2cffeec369526271 Jul 07 11:16:34.966401 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:34.966366 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-2chlh" event={"ID":"1895fd42-75c7-4daf-8e26-07d5adae8df0","Type":"ContainerStarted","Data":"14348f028fe581d89f5a481c0b1bd8e12171ff4150708c1f2cffeec369526271"} Jul 07 11:16:40.991570 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:40.991291 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8cpdb" event={"ID":"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf","Type":"ContainerStarted","Data":"36b6a6105676c5bebbeb47a90d802ce3e5b9ddaa3f37e0416dbe2a901c458251"} Jul 07 11:16:40.992195 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:40.991621 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:40.992195 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:40.991650 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8cpdb" event={"ID":"7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf","Type":"ContainerStarted","Data":"9f86d3e8204436fd23e479e28f579684bc9accaf41786af9f3ad9e116704cca4"} Jul 07 11:16:40.992933 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:40.992896 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8vcl" event={"ID":"ed8c5e9b-6266-46f1-b394-53fc8ecc1056","Type":"ContainerStarted","Data":"14cc2d22f22a0a4dcea02216cf4e1dbff717345b5fa2d1690c846e9c77b55cf1"} Jul 07 11:16:40.994541 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:40.994519 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l6xpr" event={"ID":"44a96807-d2da-429a-9ee8-27a92649ac56","Type":"ContainerStarted","Data":"a6956baba222b88d3a0051f914bf714a038039417ef8e0c248667d03cc59a8e4"} Jul 07 11:16:40.996331 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:40.996305 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw" event={"ID":"e001e573-d466-497b-8408-9f0f7d098d57","Type":"ContainerStarted","Data":"d1ae89c81eceecbfdd945e4457c3589eeacf25b62505a405c14a8b8a805c0975"} Jul 07 11:16:40.996471 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:40.996453 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw" Jul 07 11:16:41.002164 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:41.002140 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw" Jul 07 11:16:41.010269 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:41.010225 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8cpdb" podStartSLOduration=1.6970241750000001 podStartE2EDuration="8.010209903s" totalImagesPullingTime="6.313185728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:41.008903761 +0000 UTC m=+58.596770252" watchObservedRunningTime="2026-07-07 11:16:41.010209903 +0000 UTC m=+58.598076397" Jul 07 11:16:41.027698 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:41.027646 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9bwkw" podStartSLOduration=2.159822909 podStartE2EDuration="8.027628096s" totalImagesPullingTime="5.867805187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:41.026794403 +0000 UTC m=+58.614660894" watchObservedRunningTime="2026-07-07 11:16:41.027628096 +0000 UTC m=+58.615494629" Jul 07 11:16:41.043910 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:41.043773 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l6xpr" podStartSLOduration=1.730520855 podStartE2EDuration="8.043758272s" totalImagesPullingTime="6.313237417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:41.042873708 +0000 UTC m=+58.630740200" watchObservedRunningTime="2026-07-07 11:16:41.043758272 +0000 UTC m=+58.631624756" Jul 07 11:16:42.002205 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:42.002112 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8vcl" event={"ID":"ed8c5e9b-6266-46f1-b394-53fc8ecc1056","Type":"ContainerStarted","Data":"6b10ad39b89ed10ba9ab6f9c38026c6f8984c0d7defbd39e04900831bd19a2a2"} Jul 07 11:16:44.013331 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:44.013295 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8vcl" event={"ID":"ed8c5e9b-6266-46f1-b394-53fc8ecc1056","Type":"ContainerStarted","Data":"375faffbb400aacd8b9809f2bafbce0be270201967568860eaaae8eae3b0ef6c"} Jul 07 11:16:44.036544 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:44.036499 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g8vcl" podStartSLOduration=1.239840632 podStartE2EDuration="11.03647973s" totalImagesPullingTime="9.796639098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:16:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:44.034605256 +0000 UTC m=+61.622471748" watchObservedRunningTime="2026-07-07 11:16:44.03647973 +0000 UTC m=+61.624346220" Jul 07 11:16:44.917910 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:44.917877 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sv4f" Jul 07 11:16:47.094037 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.094004 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2"] Jul 07 11:16:47.097065 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.097043 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.100375 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.100175 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jfb75"] Jul 07 11:16:47.100375 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.100197 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 11:16:47.100375 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.100206 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 11:16:47.100375 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.100215 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 11:16:47.100375 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.100339 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 11:16:47.101087 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.100936 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9p47x\"" Jul 07 11:16:47.101087 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.101022 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 11:16:47.102573 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.102555 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.105914 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.105843 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hdwhd\"" Jul 07 11:16:47.106302 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.106283 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 11:16:47.107216 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.106884 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 11:16:47.107216 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.107117 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 11:16:47.121817 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.121795 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2"] Jul 07 11:16:47.183843 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.183806 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zpsp\" (UniqueName: \"kubernetes.io/projected/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-kube-api-access-2zpsp\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.183843 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.183842 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-textfile\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.184081 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.183870 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.184081 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.183897 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr4q6\" (UniqueName: \"kubernetes.io/projected/f641be91-b0b9-498b-aba8-728d2271f9c2-kube-api-access-fr4q6\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.184081 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.183975 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.184081 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.184038 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f641be91-b0b9-498b-aba8-728d2271f9c2-root\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.184081 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.184060 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f641be91-b0b9-498b-aba8-728d2271f9c2-metrics-client-ca\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.184338 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.184135 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.184338 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.184164 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.184338 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.184185 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-tls\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.184338 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.184271 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f641be91-b0b9-498b-aba8-728d2271f9c2-sys\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.184338 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.184307 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-wtmp\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.184338 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.184336 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-accelerators-collector-config\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.285547 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285501 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.285733 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285575 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f641be91-b0b9-498b-aba8-728d2271f9c2-root\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.285733 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285627 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f641be91-b0b9-498b-aba8-728d2271f9c2-metrics-client-ca\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.285733 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285642 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f641be91-b0b9-498b-aba8-728d2271f9c2-root\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.285733 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285688 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.285733 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285715 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.285982 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285739 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-tls\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.285982 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285797 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f641be91-b0b9-498b-aba8-728d2271f9c2-sys\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.285982 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285828 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-wtmp\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.285982 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:47.285848 2585 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 07 11:16:47.285982 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285856 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-accelerators-collector-config\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.285982 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285897 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zpsp\" (UniqueName: \"kubernetes.io/projected/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-kube-api-access-2zpsp\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.285982 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:47.285908 2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 11:16:47.285982 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:47.285918 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-openshift-state-metrics-tls podName:c6bab654-ad53-4cfa-8e0b-2cf60be6d410 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:47.785895689 +0000 UTC m=+65.373762160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-5cnh2" (UID: "c6bab654-ad53-4cfa-8e0b-2cf60be6d410") : secret "openshift-state-metrics-tls" not found Jul 07 11:16:47.285982 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285950 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-textfile\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.285982 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:16:47.285962 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-tls podName:f641be91-b0b9-498b-aba8-728d2271f9c2 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:47.785944733 +0000 UTC m=+65.373811203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-tls") pod "node-exporter-jfb75" (UID: "f641be91-b0b9-498b-aba8-728d2271f9c2") : secret "node-exporter-tls" not found Jul 07 11:16:47.286480 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.285997 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.286480 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.286028 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr4q6\" (UniqueName: \"kubernetes.io/projected/f641be91-b0b9-498b-aba8-728d2271f9c2-kube-api-access-fr4q6\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.286480 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.286199 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-textfile\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.286480 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.286297 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f641be91-b0b9-498b-aba8-728d2271f9c2-metrics-client-ca\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.286480 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.286330 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f641be91-b0b9-498b-aba8-728d2271f9c2-sys\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.286480 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.286458 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-wtmp\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.286813 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.286791 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-accelerators-collector-config\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.286853 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.286839 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.288878 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.288859 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.289068 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.289051 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.297661 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.297622 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr4q6\" (UniqueName: \"kubernetes.io/projected/f641be91-b0b9-498b-aba8-728d2271f9c2-kube-api-access-fr4q6\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.299875 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.299847 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zpsp\" (UniqueName: \"kubernetes.io/projected/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-kube-api-access-2zpsp\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.790693 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.790659 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:47.790693 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.790703 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-tls\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.793400 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.793370 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f641be91-b0b9-498b-aba8-728d2271f9c2-node-exporter-tls\") pod \"node-exporter-jfb75\" (UID: \"f641be91-b0b9-498b-aba8-728d2271f9c2\") " pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:47.793535 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:47.793425 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6bab654-ad53-4cfa-8e0b-2cf60be6d410-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-5cnh2\" (UID: \"c6bab654-ad53-4cfa-8e0b-2cf60be6d410\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:48.009471 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.009435 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" Jul 07 11:16:48.016056 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.016038 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jfb75" Jul 07 11:16:48.031173 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.031140 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:16:48.034032 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.034013 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.037138 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.037114 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 11:16:48.037253 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.037189 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 11:16:48.037253 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.037215 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 11:16:48.037434 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.037407 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 11:16:48.037551 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.037458 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 11:16:48.037551 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.037479 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 11:16:48.037740 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.037717 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 11:16:48.037824 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.037752 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 11:16:48.037824 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.037717 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-znmzg\"" Jul 07 11:16:48.037942 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.037723 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 11:16:48.055809 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.055784 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:16:48.092661 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.092622 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.092848 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.092673 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-volume\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.092848 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.092700 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-out\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.092848 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.092762 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-web-config\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.092848 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.092807 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.093067 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.092853 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.093067 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.092880 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.093067 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.092909 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.093067 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.092928 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6s5l\" (UniqueName: \"kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-kube-api-access-d6s5l\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.093067 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.093029 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.093311 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.093086 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.093311 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.093114 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.093311 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.093179 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.193630 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.193589 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194097 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.193641 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194097 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.193673 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194097 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.193697 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s5l\" (UniqueName: \"kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-kube-api-access-d6s5l\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194097 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.193795 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194097 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.193860 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194097 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.193891 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194097 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.193932 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194097 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.194026 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194535 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.194164 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-volume\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194535 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.194199 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-out\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194535 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.194225 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-web-config\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194535 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.194252 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.194535 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.194401 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.195162 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.195107 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.195360 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.195314 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.197233 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.197210 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.197486 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.197463 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-out\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.198020 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.197995 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.198020 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.198011 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.198194 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.198166 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.198560 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.198516 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-volume\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.198663 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.198558 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.198774 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.198740 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.199951 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.199934 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-web-config\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.203814 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.203784 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6s5l\" (UniqueName: \"kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-kube-api-access-d6s5l\") pod \"alertmanager-main-0\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.294983 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.294895 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:48.294983 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.294938 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:48.297910 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.297879 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 11:16:48.297910 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.297879 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 11:16:48.308067 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.308040 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45bf26b3-aacb-4588-bf31-04a417518c64-original-pull-secret\") pod \"global-pull-secret-syncer-5zc94\" (UID: \"45bf26b3-aacb-4588-bf31-04a417518c64\") " pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:48.308199 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.308067 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5da61a2d-4113-4869-99bf-f0d24e5edf97-metrics-certs\") pod \"network-metrics-daemon-9rwbg\" (UID: \"5da61a2d-4113-4869-99bf-f0d24e5edf97\") " pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:48.344824 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.344783 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:16:48.394814 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.394777 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5zc94" Jul 07 11:16:48.396193 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.396165 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cjbm\" (UniqueName: \"kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm\") pod \"network-check-target-kzrqj\" (UID: \"617abf3b-a72a-4600-8764-60aab5668bcd\") " pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:48.399229 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.399213 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 11:16:48.409370 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.409338 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 11:16:48.419775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.419751 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cjbm\" (UniqueName: \"kubernetes.io/projected/617abf3b-a72a-4600-8764-60aab5668bcd-kube-api-access-5cjbm\") pod \"network-check-target-kzrqj\" (UID: \"617abf3b-a72a-4600-8764-60aab5668bcd\") " pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:48.422002 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.421978 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89jkb\"" Jul 07 11:16:48.429647 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.429626 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rwbg" Jul 07 11:16:48.721243 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.721213 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42fbv\"" Jul 07 11:16:48.729215 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:48.729191 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:16:51.005375 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.005326 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8cpdb" Jul 07 11:16:51.707646 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.707616 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7ccddd76f4-g99h5"] Jul 07 11:16:51.727464 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.727436 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7ccddd76f4-g99h5"] Jul 07 11:16:51.727641 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.727546 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.731770 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.731542 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 11:16:51.731770 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.731550 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 11:16:51.731770 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.731586 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-t4l6t\"" Jul 07 11:16:51.731770 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.731550 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4dvu82m1t8svi\"" Jul 07 11:16:51.731770 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.731630 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 11:16:51.731770 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.731555 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 11:16:51.824811 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.824771 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5caa2274-4ee6-4a35-839a-8b0840478ab6-metrics-server-audit-profiles\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.824968 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.824821 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5caa2274-4ee6-4a35-839a-8b0840478ab6-audit-log\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.824968 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.824850 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5caa2274-4ee6-4a35-839a-8b0840478ab6-secret-metrics-server-tls\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.824968 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.824902 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5caa2274-4ee6-4a35-839a-8b0840478ab6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.824968 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.824929 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z89kr\" (UniqueName: \"kubernetes.io/projected/5caa2274-4ee6-4a35-839a-8b0840478ab6-kube-api-access-z89kr\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.824968 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.824964 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5caa2274-4ee6-4a35-839a-8b0840478ab6-secret-metrics-server-client-certs\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.825185 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.824982 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5caa2274-4ee6-4a35-839a-8b0840478ab6-client-ca-bundle\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.926053 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.926015 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5caa2274-4ee6-4a35-839a-8b0840478ab6-metrics-server-audit-profiles\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.926222 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.926067 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5caa2274-4ee6-4a35-839a-8b0840478ab6-audit-log\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.926222 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.926097 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5caa2274-4ee6-4a35-839a-8b0840478ab6-secret-metrics-server-tls\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.926222 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.926138 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5caa2274-4ee6-4a35-839a-8b0840478ab6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.926222 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.926178 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z89kr\" (UniqueName: \"kubernetes.io/projected/5caa2274-4ee6-4a35-839a-8b0840478ab6-kube-api-access-z89kr\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.926222 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.926210 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5caa2274-4ee6-4a35-839a-8b0840478ab6-secret-metrics-server-client-certs\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.926507 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.926233 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5caa2274-4ee6-4a35-839a-8b0840478ab6-client-ca-bundle\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.926595 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.926575 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5caa2274-4ee6-4a35-839a-8b0840478ab6-audit-log\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.927253 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.927202 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5caa2274-4ee6-4a35-839a-8b0840478ab6-metrics-server-audit-profiles\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.927253 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.927216 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5caa2274-4ee6-4a35-839a-8b0840478ab6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.929128 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.929103 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5caa2274-4ee6-4a35-839a-8b0840478ab6-secret-metrics-server-tls\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.929488 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.929468 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5caa2274-4ee6-4a35-839a-8b0840478ab6-client-ca-bundle\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.929581 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.929532 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5caa2274-4ee6-4a35-839a-8b0840478ab6-secret-metrics-server-client-certs\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:51.936305 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:51.936287 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z89kr\" (UniqueName: \"kubernetes.io/projected/5caa2274-4ee6-4a35-839a-8b0840478ab6-kube-api-access-z89kr\") pod \"metrics-server-7ccddd76f4-g99h5\" (UID: \"5caa2274-4ee6-4a35-839a-8b0840478ab6\") " pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:52.037718 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:52.037636 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:16:53.684855 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:53.683113 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5zc94"] Jul 07 11:16:53.696891 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:53.696867 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kzrqj"] Jul 07 11:16:53.699782 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:16:53.699746 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617abf3b_a72a_4600_8764_60aab5668bcd.slice/crio-d4245f9a3e729fce0738ab9c023c6d0650994e5cebf0b27e5b9d6c05466a9550 WatchSource:0}: Error finding container d4245f9a3e729fce0738ab9c023c6d0650994e5cebf0b27e5b9d6c05466a9550: Status 404 returned error can't find the container with id d4245f9a3e729fce0738ab9c023c6d0650994e5cebf0b27e5b9d6c05466a9550 Jul 07 11:16:53.937517 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:53.937383 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2"] Jul 07 11:16:53.939740 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:53.939701 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9rwbg"] Jul 07 11:16:53.940563 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:53.940544 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7ccddd76f4-g99h5"] Jul 07 11:16:53.942646 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:16:53.942617 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6bab654_ad53_4cfa_8e0b_2cf60be6d410.slice/crio-6ad895a5ef11cf882452768072cd5f949b76f64977c6794d2e1f13a0dab4405e WatchSource:0}: Error finding container 6ad895a5ef11cf882452768072cd5f949b76f64977c6794d2e1f13a0dab4405e: Status 404 returned error can't find the container with id 6ad895a5ef11cf882452768072cd5f949b76f64977c6794d2e1f13a0dab4405e Jul 07 11:16:53.943332 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:16:53.943296 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da61a2d_4113_4869_99bf_f0d24e5edf97.slice/crio-30131d7a6dbfadb96b3ec2fb0deadd1736d9e5b536b14ecdc76961990e0584ce WatchSource:0}: Error finding container 30131d7a6dbfadb96b3ec2fb0deadd1736d9e5b536b14ecdc76961990e0584ce: Status 404 returned error can't find the container with id 30131d7a6dbfadb96b3ec2fb0deadd1736d9e5b536b14ecdc76961990e0584ce Jul 07 11:16:53.944425 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:16:53.944271 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5caa2274_4ee6_4a35_839a_8b0840478ab6.slice/crio-fe0ecf49cef4a9e7b35de38a88866adca63cd512ac07792a7d34bb338b615f4c WatchSource:0}: Error finding container fe0ecf49cef4a9e7b35de38a88866adca63cd512ac07792a7d34bb338b615f4c: Status 404 returned error can't find the container with id fe0ecf49cef4a9e7b35de38a88866adca63cd512ac07792a7d34bb338b615f4c Jul 07 11:16:53.947510 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:53.947492 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:16:53.952456 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:16:53.952429 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc476bf5c_0ec5_455c_bc40_ef66fa44545a.slice/crio-6651139abca5a41beffc4c768863df8c0cb0d47c0e95d9081f57a2cfc722951b WatchSource:0}: Error finding container 6651139abca5a41beffc4c768863df8c0cb0d47c0e95d9081f57a2cfc722951b: Status 404 returned error can't find the container with id 6651139abca5a41beffc4c768863df8c0cb0d47c0e95d9081f57a2cfc722951b Jul 07 11:16:54.050286 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.049827 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-2chlh" event={"ID":"1895fd42-75c7-4daf-8e26-07d5adae8df0","Type":"ContainerStarted","Data":"e9c5648bd8307470a4d750294efcd28aa7127de0ada614fb918fb2419075a4cb"} Jul 07 11:16:54.050286 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.050236 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-2chlh" Jul 07 11:16:54.052118 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.051992 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" event={"ID":"c6bab654-ad53-4cfa-8e0b-2cf60be6d410","Type":"ContainerStarted","Data":"671c152d00a7b462422a06213476af98e9fc63b7b831d2549610b03c856e9f02"} Jul 07 11:16:54.052118 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.052046 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" event={"ID":"c6bab654-ad53-4cfa-8e0b-2cf60be6d410","Type":"ContainerStarted","Data":"6ad895a5ef11cf882452768072cd5f949b76f64977c6794d2e1f13a0dab4405e"} Jul 07 11:16:54.053507 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.053483 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" event={"ID":"5caa2274-4ee6-4a35-839a-8b0840478ab6","Type":"ContainerStarted","Data":"fe0ecf49cef4a9e7b35de38a88866adca63cd512ac07792a7d34bb338b615f4c"} Jul 07 11:16:54.054896 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.054870 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9rwbg" event={"ID":"5da61a2d-4113-4869-99bf-f0d24e5edf97","Type":"ContainerStarted","Data":"30131d7a6dbfadb96b3ec2fb0deadd1736d9e5b536b14ecdc76961990e0584ce"} Jul 07 11:16:54.056068 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.056042 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5zc94" event={"ID":"45bf26b3-aacb-4588-bf31-04a417518c64","Type":"ContainerStarted","Data":"4ca44b2bdaacd5e818fccd292300faf911c7fa910f9b2d048733f31f608160de"} Jul 07 11:16:54.057385 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.057339 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerStarted","Data":"6651139abca5a41beffc4c768863df8c0cb0d47c0e95d9081f57a2cfc722951b"} Jul 07 11:16:54.058619 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.058599 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kzrqj" event={"ID":"617abf3b-a72a-4600-8764-60aab5668bcd","Type":"ContainerStarted","Data":"d4245f9a3e729fce0738ab9c023c6d0650994e5cebf0b27e5b9d6c05466a9550"} Jul 07 11:16:54.059882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.059855 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jfb75" event={"ID":"f641be91-b0b9-498b-aba8-728d2271f9c2","Type":"ContainerStarted","Data":"99b5282795f054dbc3a172e3a9469f57801a2497c54300ba040799cc90ff6bac"} Jul 07 11:16:54.065239 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.065221 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-2chlh" Jul 07 11:16:54.068858 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.068812 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-2chlh" podStartSLOduration=1.780728915 podStartE2EDuration="21.068796041s" totalImagesPullingTime="19.288067126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:54.068009279 +0000 UTC m=+71.655875771" watchObservedRunningTime="2026-07-07 11:16:54.068796041 +0000 UTC m=+71.656662534" Jul 07 11:16:54.978397 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:54.977839 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bb544648c-4lkkg" Jul 07 11:16:55.074145 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:16:55.074074 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" event={"ID":"c6bab654-ad53-4cfa-8e0b-2cf60be6d410","Type":"ContainerStarted","Data":"c9b60a8cb78bf70aeb1a1ceeaebe7e12433384a54c6ff501c5fa47703dfa61c9"} Jul 07 11:17:03.110287 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.110248 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" event={"ID":"c6bab654-ad53-4cfa-8e0b-2cf60be6d410","Type":"ContainerStarted","Data":"05c85dac0b7f1d248a7a51252ac2a3d0deecbfb3b57a6834771e72570cc83ffe"} Jul 07 11:17:03.111783 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.111753 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" event={"ID":"5caa2274-4ee6-4a35-839a-8b0840478ab6","Type":"ContainerStarted","Data":"0d1e91ee83a8b0822ae290883616a2b34a320d57e400e51d7ea7f75859fbed4e"} Jul 07 11:17:03.113401 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.113380 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9rwbg" event={"ID":"5da61a2d-4113-4869-99bf-f0d24e5edf97","Type":"ContainerStarted","Data":"a356fc436f5218f9d4fbb6fe35e2892c0d896a2317a07a5d26d5615c47ddb20d"} Jul 07 11:17:03.113505 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.113426 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9rwbg" event={"ID":"5da61a2d-4113-4869-99bf-f0d24e5edf97","Type":"ContainerStarted","Data":"3974f1446a98121974031e569533eac589f8b7244a1bb6f6ee5c3d6aa86111a3"} Jul 07 11:17:03.114836 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.114813 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5zc94" event={"ID":"45bf26b3-aacb-4588-bf31-04a417518c64","Type":"ContainerStarted","Data":"7fe53d02a7fd8b84a9a03394dc75b1cb9c1504021b6ee124d0738c7e602b0c16"} Jul 07 11:17:03.116155 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.116135 2585 generic.go:356] "Generic (PLEG): container finished" podID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerID="5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f" exitCode=0 Jul 07 11:17:03.116240 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.116204 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerDied","Data":"5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f"} Jul 07 11:17:03.117699 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.117681 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kzrqj" event={"ID":"617abf3b-a72a-4600-8764-60aab5668bcd","Type":"ContainerStarted","Data":"08d5d58efc72c52d7ba682cf3f83f39b522a085e5503bfe1f95626ef94991836"} Jul 07 11:17:03.117760 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.117732 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:17:03.119146 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.119128 2585 generic.go:356] "Generic (PLEG): container finished" podID="f641be91-b0b9-498b-aba8-728d2271f9c2" containerID="c5c773cf3dba40e548dc5583b1347862d6772a8c837a2f84a555e30edb655af3" exitCode=0 Jul 07 11:17:03.119220 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.119164 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jfb75" event={"ID":"f641be91-b0b9-498b-aba8-728d2271f9c2","Type":"ContainerDied","Data":"c5c773cf3dba40e548dc5583b1347862d6772a8c837a2f84a555e30edb655af3"} Jul 07 11:17:03.167569 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.167526 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5cnh2" podStartSLOduration=8.201043405 podStartE2EDuration="16.167510594s" totalImagesPullingTime="7.966467189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:03.16692455 +0000 UTC m=+80.754791045" watchObservedRunningTime="2026-07-07 11:17:03.167510594 +0000 UTC m=+80.755377083" Jul 07 11:17:03.204459 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.204413 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kzrqj" podStartSLOduration=70.647367032 podStartE2EDuration="1m19.20439652s" totalImagesPullingTime="8.557029488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:03.199677441 +0000 UTC m=+80.787543930" watchObservedRunningTime="2026-07-07 11:17:03.20439652 +0000 UTC m=+80.792263010" Jul 07 11:17:03.339009 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.338925 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5zc94" podStartSLOduration=70.756711526 podStartE2EDuration="1m19.338908431s" totalImagesPullingTime="8.582196905s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:03.338701105 +0000 UTC m=+80.926567595" watchObservedRunningTime="2026-07-07 11:17:03.338908431 +0000 UTC m=+80.926774920" Jul 07 11:17:03.339687 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.339647 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9rwbg" podStartSLOduration=71.027023254 podStartE2EDuration="1m19.339634228s" totalImagesPullingTime="8.312610974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:03.292275008 +0000 UTC m=+80.880141499" watchObservedRunningTime="2026-07-07 11:17:03.339634228 +0000 UTC m=+80.927500717" Jul 07 11:17:03.433929 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:03.433874 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" podStartSLOduration=4.122877893 podStartE2EDuration="12.43385786s" totalImagesPullingTime="8.310979967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:03.370825802 +0000 UTC m=+80.958692291" watchObservedRunningTime="2026-07-07 11:17:03.43385786 +0000 UTC m=+81.021724354" Jul 07 11:17:04.126148 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:04.126106 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jfb75" event={"ID":"f641be91-b0b9-498b-aba8-728d2271f9c2","Type":"ContainerStarted","Data":"7d0fdf3ca409832f99376bd2a193b66cfc73aa9eddb5018758c9c97573d80421"} Jul 07 11:17:04.126638 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:04.126175 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jfb75" event={"ID":"f641be91-b0b9-498b-aba8-728d2271f9c2","Type":"ContainerStarted","Data":"22b6bd24cb3e5534f7de4fee9266abf3130db1dda0c005dd7fea235a06b2c898"} Jul 07 11:17:04.163522 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:04.163472 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jfb75" podStartSLOduration=8.496305539 podStartE2EDuration="17.163455492s" totalImagesPullingTime="8.543341964s" totalInitContainerRuntime="123.807989ms" isStatefulPod=true podCreationTimestamp="2026-07-07 11:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:04.16256953 +0000 UTC m=+81.750436025" watchObservedRunningTime="2026-07-07 11:17:04.163455492 +0000 UTC m=+81.751321982" Jul 07 11:17:06.137178 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:06.137145 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerStarted","Data":"b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761"} Jul 07 11:17:06.137638 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:06.137211 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerStarted","Data":"adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b"} Jul 07 11:17:06.137638 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:06.137237 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerStarted","Data":"8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3"} Jul 07 11:17:06.137638 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:06.137262 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerStarted","Data":"d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0"} Jul 07 11:17:06.137638 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:06.137283 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerStarted","Data":"5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045"} Jul 07 11:17:08.147711 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:08.147679 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerStarted","Data":"4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904"} Jul 07 11:17:08.196684 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:08.196632 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=8.400663261 podStartE2EDuration="20.196615808s" totalImagesPullingTime="11.774442489s" totalInitContainerRuntime="21.510058ms" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:08.195850782 +0000 UTC m=+85.783717272" watchObservedRunningTime="2026-07-07 11:17:08.196615808 +0000 UTC m=+85.784482297" Jul 07 11:17:12.038363 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:12.038302 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:17:12.038849 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:12.038397 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:17:32.043827 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:32.043795 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:17:32.047952 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:32.047928 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7ccddd76f4-g99h5" Jul 07 11:17:34.128287 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:34.128254 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kzrqj" Jul 07 11:17:37.413110 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:37.413075 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:17:37.413615 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:37.413459 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="alertmanager" containerID="cri-o://5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045" gracePeriod=120 Jul 07 11:17:37.413615 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:37.413523 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy-metric" containerID="cri-o://b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761" gracePeriod=120 Jul 07 11:17:37.413615 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:37.413545 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="prom-label-proxy" containerID="cri-o://4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904" gracePeriod=120 Jul 07 11:17:37.413775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:37.413598 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="config-reloader" containerID="cri-o://d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0" gracePeriod=120 Jul 07 11:17:37.413775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:37.413557 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy-web" containerID="cri-o://8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3" gracePeriod=120 Jul 07 11:17:37.413775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:37.413598 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy" containerID="cri-o://adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b" gracePeriod=120 Jul 07 11:17:38.255105 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.255070 2585 generic.go:356] "Generic (PLEG): container finished" podID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerID="4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904" exitCode=0 Jul 07 11:17:38.255105 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.255097 2585 generic.go:356] "Generic (PLEG): container finished" podID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerID="adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b" exitCode=0 Jul 07 11:17:38.255105 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.255103 2585 generic.go:356] "Generic (PLEG): container finished" podID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerID="d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0" exitCode=0 Jul 07 11:17:38.255105 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.255109 2585 generic.go:356] "Generic (PLEG): container finished" podID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerID="5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045" exitCode=0 Jul 07 11:17:38.255467 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.255143 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerDied","Data":"4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904"} Jul 07 11:17:38.255467 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.255228 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerDied","Data":"adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b"} Jul 07 11:17:38.255467 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.255259 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerDied","Data":"d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0"} Jul 07 11:17:38.255467 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.255287 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerDied","Data":"5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045"} Jul 07 11:17:38.707973 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.707947 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:38.819454 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819326 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-main-db\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819454 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819388 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819454 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819413 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-tls-assets\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819454 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819439 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-cluster-tls-config\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819463 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6s5l\" (UniqueName: \"kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-kube-api-access-d6s5l\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819494 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-volume\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819522 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-web-config\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819554 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-out\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819584 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-metrics-client-ca\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819612 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-main-tls\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819637 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-web\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819665 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819699 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-trusted-ca-bundle\") pod \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\" (UID: \"c476bf5c-0ec5-455c-bc40-ef66fa44545a\") " Jul 07 11:17:38.819775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819699 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:17:38.820271 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.819927 2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-main-db\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.820271 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.820060 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:38.820271 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.820260 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:38.822995 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.822963 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-out" (OuterVolumeSpecName: "config-out") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:17:38.823141 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.823076 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:38.823428 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.823379 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:17:38.823520 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.823456 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:38.823631 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.823606 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-kube-api-access-d6s5l" (OuterVolumeSpecName: "kube-api-access-d6s5l") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "kube-api-access-d6s5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:17:38.823631 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.823609 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-volume" (OuterVolumeSpecName: "config-volume") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:38.823812 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.823724 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:38.823926 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.823905 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:38.826588 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.826567 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:38.832334 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.832311 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-web-config" (OuterVolumeSpecName: "web-config") pod "c476bf5c-0ec5-455c-bc40-ef66fa44545a" (UID: "c476bf5c-0ec5-455c-bc40-ef66fa44545a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:38.920966 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.920930 2585 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-volume\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.920966 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.920963 2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-web-config\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.920966 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.920972 2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c476bf5c-0ec5-455c-bc40-ef66fa44545a-config-out\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.921191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.920980 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-metrics-client-ca\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.921191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.920992 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-main-tls\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.921191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.921002 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.921191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.921011 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.921191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.921020 2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c476bf5c-0ec5-455c-bc40-ef66fa44545a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.921191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.921029 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.921191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.921037 2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-tls-assets\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.921191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.921046 2585 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c476bf5c-0ec5-455c-bc40-ef66fa44545a-cluster-tls-config\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:38.921191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:38.921054 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6s5l\" (UniqueName: \"kubernetes.io/projected/c476bf5c-0ec5-455c-bc40-ef66fa44545a-kube-api-access-d6s5l\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:17:39.261892 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.261860 2585 generic.go:356] "Generic (PLEG): container finished" podID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerID="b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761" exitCode=0 Jul 07 11:17:39.261892 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.261888 2585 generic.go:356] "Generic (PLEG): container finished" podID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerID="8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3" exitCode=0 Jul 07 11:17:39.262088 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.261935 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.262088 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.261951 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerDied","Data":"b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761"} Jul 07 11:17:39.262088 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.262018 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerDied","Data":"8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3"} Jul 07 11:17:39.262088 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.262048 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c476bf5c-0ec5-455c-bc40-ef66fa44545a","Type":"ContainerDied","Data":"6651139abca5a41beffc4c768863df8c0cb0d47c0e95d9081f57a2cfc722951b"} Jul 07 11:17:39.262209 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.262094 2585 scope.go:117] "RemoveContainer" containerID="4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904" Jul 07 11:17:39.270394 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.270373 2585 scope.go:117] "RemoveContainer" containerID="b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761" Jul 07 11:17:39.277767 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.277751 2585 scope.go:117] "RemoveContainer" containerID="adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b" Jul 07 11:17:39.284681 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.284665 2585 scope.go:117] "RemoveContainer" containerID="8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3" Jul 07 11:17:39.289950 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.289928 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:17:39.292259 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.292237 2585 scope.go:117] "RemoveContainer" containerID="d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0" Jul 07 11:17:39.296100 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.296081 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:17:39.299714 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.299693 2585 scope.go:117] "RemoveContainer" containerID="5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045" Jul 07 11:17:39.306643 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.306627 2585 scope.go:117] "RemoveContainer" containerID="5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f" Jul 07 11:17:39.313012 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.312998 2585 scope.go:117] "RemoveContainer" containerID="4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904" Jul 07 11:17:39.313287 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:17:39.313259 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904\": container with ID starting with 4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904 not found: ID does not exist" containerID="4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904" Jul 07 11:17:39.313334 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.313297 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904"} err="failed to get container status \"4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904\": rpc error: code = NotFound desc = could not find container \"4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904\": container with ID starting with 4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904 not found: ID does not exist" Jul 07 11:17:39.313334 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.313331 2585 scope.go:117] "RemoveContainer" containerID="b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761" Jul 07 11:17:39.313570 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:17:39.313557 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761\": container with ID starting with b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761 not found: ID does not exist" containerID="b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761" Jul 07 11:17:39.313612 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.313575 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761"} err="failed to get container status \"b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761\": rpc error: code = NotFound desc = could not find container \"b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761\": container with ID starting with b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761 not found: ID does not exist" Jul 07 11:17:39.313612 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.313587 2585 scope.go:117] "RemoveContainer" containerID="adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b" Jul 07 11:17:39.313762 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:17:39.313748 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b\": container with ID starting with adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b not found: ID does not exist" containerID="adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b" Jul 07 11:17:39.313805 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.313764 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b"} err="failed to get container status \"adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b\": rpc error: code = NotFound desc = could not find container \"adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b\": container with ID starting with adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b not found: ID does not exist" Jul 07 11:17:39.313805 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.313776 2585 scope.go:117] "RemoveContainer" containerID="8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3" Jul 07 11:17:39.313959 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:17:39.313939 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3\": container with ID starting with 8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3 not found: ID does not exist" containerID="8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3" Jul 07 11:17:39.314055 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.313960 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3"} err="failed to get container status \"8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3\": rpc error: code = NotFound desc = could not find container \"8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3\": container with ID starting with 8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3 not found: ID does not exist" Jul 07 11:17:39.314055 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.313978 2585 scope.go:117] "RemoveContainer" containerID="d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0" Jul 07 11:17:39.314193 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:17:39.314173 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0\": container with ID starting with d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0 not found: ID does not exist" containerID="d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0" Jul 07 11:17:39.314235 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.314200 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0"} err="failed to get container status \"d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0\": rpc error: code = NotFound desc = could not find container \"d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0\": container with ID starting with d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0 not found: ID does not exist" Jul 07 11:17:39.314235 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.314217 2585 scope.go:117] "RemoveContainer" containerID="5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045" Jul 07 11:17:39.314450 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:17:39.314435 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045\": container with ID starting with 5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045 not found: ID does not exist" containerID="5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045" Jul 07 11:17:39.314487 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.314454 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045"} err="failed to get container status \"5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045\": rpc error: code = NotFound desc = could not find container \"5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045\": container with ID starting with 5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045 not found: ID does not exist" Jul 07 11:17:39.314487 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.314467 2585 scope.go:117] "RemoveContainer" containerID="5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f" Jul 07 11:17:39.314695 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:17:39.314676 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f\": container with ID starting with 5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f not found: ID does not exist" containerID="5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f" Jul 07 11:17:39.314771 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.314703 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f"} err="failed to get container status \"5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f\": rpc error: code = NotFound desc = could not find container \"5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f\": container with ID starting with 5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f not found: ID does not exist" Jul 07 11:17:39.314771 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.314719 2585 scope.go:117] "RemoveContainer" containerID="4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904" Jul 07 11:17:39.314974 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.314956 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904"} err="failed to get container status \"4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904\": rpc error: code = NotFound desc = could not find container \"4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904\": container with ID starting with 4b8bf86be7df47534cbbd8d5a20953818779f02ac17486a2700b1b8f42c94904 not found: ID does not exist" Jul 07 11:17:39.315019 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.314976 2585 scope.go:117] "RemoveContainer" containerID="b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761" Jul 07 11:17:39.315206 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.315191 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761"} err="failed to get container status \"b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761\": rpc error: code = NotFound desc = could not find container \"b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761\": container with ID starting with b302bfc0c6a308e4179100e744721b46c159ba15ebb354cc2b93601298c18761 not found: ID does not exist" Jul 07 11:17:39.315256 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.315208 2585 scope.go:117] "RemoveContainer" containerID="adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b" Jul 07 11:17:39.315445 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.315425 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b"} err="failed to get container status \"adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b\": rpc error: code = NotFound desc = could not find container \"adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b\": container with ID starting with adbf3d3b2ae66c67bda0cfd044584d571ac4e37fac5a9c4a646c71665769652b not found: ID does not exist" Jul 07 11:17:39.315445 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.315444 2585 scope.go:117] "RemoveContainer" containerID="8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3" Jul 07 11:17:39.315703 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.315673 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3"} err="failed to get container status \"8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3\": rpc error: code = NotFound desc = could not find container \"8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3\": container with ID starting with 8e40f9482c4ce52511b1b486c817d2649aff67bad6b9cae642723a8bc6e67ce3 not found: ID does not exist" Jul 07 11:17:39.315703 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.315702 2585 scope.go:117] "RemoveContainer" containerID="d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0" Jul 07 11:17:39.315949 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.315932 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0"} err="failed to get container status \"d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0\": rpc error: code = NotFound desc = could not find container \"d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0\": container with ID starting with d4d37459612378bc208c901c41b23ef5fe2ac973e76fd4bde9a3ccd6dcde29e0 not found: ID does not exist" Jul 07 11:17:39.315997 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.315952 2585 scope.go:117] "RemoveContainer" containerID="5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045" Jul 07 11:17:39.316169 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.316155 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045"} err="failed to get container status \"5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045\": rpc error: code = NotFound desc = could not find container \"5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045\": container with ID starting with 5f46ac0e3dd72c932c189681beab0946f001a6c9bfb3330fcf7131bd56a02045 not found: ID does not exist" Jul 07 11:17:39.316169 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.316168 2585 scope.go:117] "RemoveContainer" containerID="5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f" Jul 07 11:17:39.316415 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.316393 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f"} err="failed to get container status \"5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f\": rpc error: code = NotFound desc = could not find container \"5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f\": container with ID starting with 5c3cd1d1951631a4b24ecfbc4e740c32407eb97f5734d5e30a803868bfd4913f not found: ID does not exist" Jul 07 11:17:39.327120 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327096 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:17:39.327371 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327359 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy-web" Jul 07 11:17:39.327371 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327371 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy-web" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327381 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327387 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327399 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="init-config-reloader" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327404 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="init-config-reloader" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327412 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="prom-label-proxy" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327417 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="prom-label-proxy" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327423 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="alertmanager" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327428 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="alertmanager" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327441 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="config-reloader" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327446 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="config-reloader" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327452 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy-metric" Jul 07 11:17:39.327457 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327457 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy-metric" Jul 07 11:17:39.327788 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327497 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="alertmanager" Jul 07 11:17:39.327788 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327504 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy" Jul 07 11:17:39.327788 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327512 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="config-reloader" Jul 07 11:17:39.327788 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327518 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy-web" Jul 07 11:17:39.327788 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327524 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="kube-rbac-proxy-metric" Jul 07 11:17:39.327788 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.327530 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" containerName="prom-label-proxy" Jul 07 11:17:39.354098 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.354058 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:17:39.354238 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.354169 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.357577 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.357528 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 11:17:39.357716 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.357528 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 11:17:39.357716 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.357682 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 11:17:39.357837 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.357796 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-znmzg\"" Jul 07 11:17:39.357890 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.357833 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 11:17:39.358064 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.358047 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 11:17:39.358176 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.358083 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 11:17:39.358176 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.358092 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 11:17:39.358287 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.358162 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 11:17:39.363420 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.363400 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 11:17:39.424646 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424619 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-web-config\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.424646 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424651 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76530572-a851-43b8-a567-d586360a5e8f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.424853 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424674 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-config-volume\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.424853 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424706 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hljvx\" (UniqueName: \"kubernetes.io/projected/76530572-a851-43b8-a567-d586360a5e8f-kube-api-access-hljvx\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.424853 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424749 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76530572-a851-43b8-a567-d586360a5e8f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.424853 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424790 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.424853 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424808 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/76530572-a851-43b8-a567-d586360a5e8f-config-out\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.425017 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424859 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.425017 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424876 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/76530572-a851-43b8-a567-d586360a5e8f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.425017 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424892 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.425017 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424913 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.425017 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424951 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/76530572-a851-43b8-a567-d586360a5e8f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.425017 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.424978 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526158 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526085 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76530572-a851-43b8-a567-d586360a5e8f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526158 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526122 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-config-volume\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526158 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526141 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hljvx\" (UniqueName: \"kubernetes.io/projected/76530572-a851-43b8-a567-d586360a5e8f-kube-api-access-hljvx\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526430 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526280 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76530572-a851-43b8-a567-d586360a5e8f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526430 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526380 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526430 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526411 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/76530572-a851-43b8-a567-d586360a5e8f-config-out\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526569 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526456 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526569 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526485 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/76530572-a851-43b8-a567-d586360a5e8f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526569 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526507 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526569 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526542 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526759 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526584 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/76530572-a851-43b8-a567-d586360a5e8f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526759 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526624 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.526759 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.526651 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-web-config\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.527290 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.527230 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76530572-a851-43b8-a567-d586360a5e8f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.527290 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.527280 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76530572-a851-43b8-a567-d586360a5e8f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.527492 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.527339 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/76530572-a851-43b8-a567-d586360a5e8f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.529498 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.529469 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-web-config\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.529746 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.529725 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/76530572-a851-43b8-a567-d586360a5e8f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.530104 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.529734 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.530104 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.529886 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/76530572-a851-43b8-a567-d586360a5e8f-config-out\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.530104 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.529931 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-config-volume\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.530104 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.529970 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.530308 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.530162 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.530308 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.530238 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.530898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.530882 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/76530572-a851-43b8-a567-d586360a5e8f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.535234 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.535215 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hljvx\" (UniqueName: \"kubernetes.io/projected/76530572-a851-43b8-a567-d586360a5e8f-kube-api-access-hljvx\") pod \"alertmanager-main-0\" (UID: \"76530572-a851-43b8-a567-d586360a5e8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.664750 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.664711 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:17:39.675279 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.675251 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c476bf5c-0ec5-455c-bc40-ef66fa44545a" path="/var/lib/kubelet/pods/c476bf5c-0ec5-455c-bc40-ef66fa44545a/volumes" Jul 07 11:17:39.834239 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:39.834159 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:17:39.839038 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:17:39.839013 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76530572_a851_43b8_a567_d586360a5e8f.slice/crio-23e2c22108d16a720a9e4634a3076215079bdba1e707ccd54a3563a8f13f2f7c WatchSource:0}: Error finding container 23e2c22108d16a720a9e4634a3076215079bdba1e707ccd54a3563a8f13f2f7c: Status 404 returned error can't find the container with id 23e2c22108d16a720a9e4634a3076215079bdba1e707ccd54a3563a8f13f2f7c Jul 07 11:17:40.267315 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:40.267278 2585 generic.go:356] "Generic (PLEG): container finished" podID="76530572-a851-43b8-a567-d586360a5e8f" containerID="7aa4a072c9c2cc7e09bb7b466ee899b27dd350b960c006e519e27cc660f75b40" exitCode=0 Jul 07 11:17:40.267468 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:40.267328 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"76530572-a851-43b8-a567-d586360a5e8f","Type":"ContainerDied","Data":"7aa4a072c9c2cc7e09bb7b466ee899b27dd350b960c006e519e27cc660f75b40"} Jul 07 11:17:40.267468 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:40.267407 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"76530572-a851-43b8-a567-d586360a5e8f","Type":"ContainerStarted","Data":"23e2c22108d16a720a9e4634a3076215079bdba1e707ccd54a3563a8f13f2f7c"} Jul 07 11:17:41.274594 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.274558 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"76530572-a851-43b8-a567-d586360a5e8f","Type":"ContainerStarted","Data":"a7e168fa76f91ce930b549b59d1364b45c7547ef020bcb173da1c7a24248b3e7"} Jul 07 11:17:41.275045 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.274649 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"76530572-a851-43b8-a567-d586360a5e8f","Type":"ContainerStarted","Data":"bb4054e8cf2c7ae2379bcdeb422bb7c187387ab68c45cab12f70a96375ead97e"} Jul 07 11:17:41.275045 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.274674 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"76530572-a851-43b8-a567-d586360a5e8f","Type":"ContainerStarted","Data":"60f3a9bf1812f514c94bc72972240f4e8c594881fd3ffcf6fe0d8c0600b443bf"} Jul 07 11:17:41.275045 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.274703 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"76530572-a851-43b8-a567-d586360a5e8f","Type":"ContainerStarted","Data":"1a3f29e674372669ae70d52c33fecea18a21c14f3b55c4de7273e57616bb9403"} Jul 07 11:17:41.275045 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.274726 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"76530572-a851-43b8-a567-d586360a5e8f","Type":"ContainerStarted","Data":"0fb24540edb7c86ceb44fb6f5b84db58e01848698f15eec87d5c74f5dd4b9ba0"} Jul 07 11:17:41.275045 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.274750 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"76530572-a851-43b8-a567-d586360a5e8f","Type":"ContainerStarted","Data":"9a0145bd48d2ce39789d0410f30559f05e602d21acb6e2e0ef089aabb97cc69b"} Jul 07 11:17:41.314374 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.314311 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.302167365 podStartE2EDuration="2.314295213s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.127848ms" isStatefulPod=false podCreationTimestamp="2026-07-07 11:17:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:41.312081978 +0000 UTC m=+118.899948468" watchObservedRunningTime="2026-07-07 11:17:41.314295213 +0000 UTC m=+118.902161757" Jul 07 11:17:41.460929 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.460897 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6f8746c6c6-8gknp"] Jul 07 11:17:41.483026 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.482993 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f8746c6c6-8gknp"] Jul 07 11:17:41.483165 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.483130 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.487412 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.487387 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 11:17:41.487412 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.487397 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 11:17:41.488016 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.487997 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bxrr4\"" Jul 07 11:17:41.488122 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.488000 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 11:17:41.488122 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.488084 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 11:17:41.488419 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.488404 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 11:17:41.494246 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.494228 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 11:17:41.543024 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.542940 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-federate-client-tls\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.543024 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.542977 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.543197 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.543030 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-secret-telemeter-client\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.543197 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.543063 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m24vn\" (UniqueName: \"kubernetes.io/projected/060ccbed-f9be-404c-8a7c-11e47e77646a-kube-api-access-m24vn\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.543197 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.543088 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060ccbed-f9be-404c-8a7c-11e47e77646a-serving-certs-ca-bundle\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.543197 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.543120 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-telemeter-client-tls\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.543197 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.543145 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/060ccbed-f9be-404c-8a7c-11e47e77646a-metrics-client-ca\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.543197 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.543171 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060ccbed-f9be-404c-8a7c-11e47e77646a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.644473 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.644434 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-federate-client-tls\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.644625 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.644478 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.644625 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.644508 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-secret-telemeter-client\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.644625 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.644530 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m24vn\" (UniqueName: \"kubernetes.io/projected/060ccbed-f9be-404c-8a7c-11e47e77646a-kube-api-access-m24vn\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.644625 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.644546 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060ccbed-f9be-404c-8a7c-11e47e77646a-serving-certs-ca-bundle\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.644625 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.644568 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-telemeter-client-tls\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.644625 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.644592 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/060ccbed-f9be-404c-8a7c-11e47e77646a-metrics-client-ca\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.644625 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.644616 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060ccbed-f9be-404c-8a7c-11e47e77646a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.645428 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.645396 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/060ccbed-f9be-404c-8a7c-11e47e77646a-metrics-client-ca\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.645428 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.645423 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060ccbed-f9be-404c-8a7c-11e47e77646a-serving-certs-ca-bundle\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.645745 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.645724 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/060ccbed-f9be-404c-8a7c-11e47e77646a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.647130 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.647104 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-federate-client-tls\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.647224 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.647164 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.647476 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.647460 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-secret-telemeter-client\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.647565 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.647548 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/060ccbed-f9be-404c-8a7c-11e47e77646a-telemeter-client-tls\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.656264 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.656240 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m24vn\" (UniqueName: \"kubernetes.io/projected/060ccbed-f9be-404c-8a7c-11e47e77646a-kube-api-access-m24vn\") pod \"telemeter-client-6f8746c6c6-8gknp\" (UID: \"060ccbed-f9be-404c-8a7c-11e47e77646a\") " pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.793248 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.793162 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" Jul 07 11:17:41.964289 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:41.964257 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f8746c6c6-8gknp"] Jul 07 11:17:41.967529 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:17:41.967499 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod060ccbed_f9be_404c_8a7c_11e47e77646a.slice/crio-a6f2b805d69f76b5f37a9ace247c78389cc62907659bb31a905137024a50cbcd WatchSource:0}: Error finding container a6f2b805d69f76b5f37a9ace247c78389cc62907659bb31a905137024a50cbcd: Status 404 returned error can't find the container with id a6f2b805d69f76b5f37a9ace247c78389cc62907659bb31a905137024a50cbcd Jul 07 11:17:42.282780 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:42.282747 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" event={"ID":"060ccbed-f9be-404c-8a7c-11e47e77646a","Type":"ContainerStarted","Data":"a6f2b805d69f76b5f37a9ace247c78389cc62907659bb31a905137024a50cbcd"} Jul 07 11:17:45.296509 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:45.296473 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" event={"ID":"060ccbed-f9be-404c-8a7c-11e47e77646a","Type":"ContainerStarted","Data":"541a34f50173d25fccc69bd02dd17cb4097a0c1e8119fe610c47633b56e9aac7"} Jul 07 11:17:45.296913 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:45.296568 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" event={"ID":"060ccbed-f9be-404c-8a7c-11e47e77646a","Type":"ContainerStarted","Data":"6faa407611b4cd6e32e758dd5cb9a0c2059a9986d1c58cb2090dc414e4229b3a"} Jul 07 11:17:45.296913 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:45.296589 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" event={"ID":"060ccbed-f9be-404c-8a7c-11e47e77646a","Type":"ContainerStarted","Data":"9e9af7708b63c68170f13c792af6bdb3b878171a7db1dc30b78eaff3e84bbe13"} Jul 07 11:17:45.329265 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:17:45.329216 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6f8746c6c6-8gknp" podStartSLOduration=2.065279329 podStartE2EDuration="4.32919754s" totalImagesPullingTime="2.263918211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:45.326552296 +0000 UTC m=+122.914418789" watchObservedRunningTime="2026-07-07 11:17:45.32919754 +0000 UTC m=+122.917064033" Jul 07 11:19:42.531155 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.531113 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7"] Jul 07 11:19:42.533523 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.533501 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:19:42.537447 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.537420 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-n9mdv\"" Jul 07 11:19:42.537812 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.537799 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 11:19:42.538476 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.538464 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 11:19:42.554441 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.554414 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7"] Jul 07 11:19:42.599302 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.599269 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:19:42.599526 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.599325 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncl2b\" (UniqueName: \"kubernetes.io/projected/09540c73-4f54-4415-99e5-06cbfc456237-kube-api-access-ncl2b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:19:42.599526 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.599451 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:19:42.700549 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.700511 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:19:42.700549 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.700559 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncl2b\" (UniqueName: \"kubernetes.io/projected/09540c73-4f54-4415-99e5-06cbfc456237-kube-api-access-ncl2b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:19:42.700815 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.700597 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:19:42.700976 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.700956 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:19:42.701043 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.700963 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:19:42.710950 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.710913 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncl2b\" (UniqueName: \"kubernetes.io/projected/09540c73-4f54-4415-99e5-06cbfc456237-kube-api-access-ncl2b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:19:42.842669 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.842570 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:19:42.972934 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:42.972904 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7"] Jul 07 11:19:42.976937 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:19:42.976912 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09540c73_4f54_4415_99e5_06cbfc456237.slice/crio-f8afd88ef74e281f23029ded72e002a539c7d74032c73d13eeb7694c500d7c02 WatchSource:0}: Error finding container f8afd88ef74e281f23029ded72e002a539c7d74032c73d13eeb7694c500d7c02: Status 404 returned error can't find the container with id f8afd88ef74e281f23029ded72e002a539c7d74032c73d13eeb7694c500d7c02 Jul 07 11:19:43.720261 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:43.720222 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" event={"ID":"09540c73-4f54-4415-99e5-06cbfc456237","Type":"ContainerStarted","Data":"f8afd88ef74e281f23029ded72e002a539c7d74032c73d13eeb7694c500d7c02"} Jul 07 11:19:51.757360 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:51.757316 2585 generic.go:356] "Generic (PLEG): container finished" podID="09540c73-4f54-4415-99e5-06cbfc456237" containerID="dc6e68b02cfa33e3a26a888bd1653cbb64508d35bcc4f2e9d728d9fdd3bf8345" exitCode=0 Jul 07 11:19:51.757754 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:51.757406 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" event={"ID":"09540c73-4f54-4415-99e5-06cbfc456237","Type":"ContainerDied","Data":"dc6e68b02cfa33e3a26a888bd1653cbb64508d35bcc4f2e9d728d9fdd3bf8345"} Jul 07 11:19:53.768460 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:53.768428 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" event={"ID":"09540c73-4f54-4415-99e5-06cbfc456237","Type":"ContainerStarted","Data":"71063c28b2e2c284e7bc1741c3df7a19b65977d1a6d1c4cbbc69ba44c4bfe165"} Jul 07 11:19:54.773625 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:54.773595 2585 generic.go:356] "Generic (PLEG): container finished" podID="09540c73-4f54-4415-99e5-06cbfc456237" containerID="71063c28b2e2c284e7bc1741c3df7a19b65977d1a6d1c4cbbc69ba44c4bfe165" exitCode=0 Jul 07 11:19:54.773981 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:19:54.773664 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" event={"ID":"09540c73-4f54-4415-99e5-06cbfc456237","Type":"ContainerDied","Data":"71063c28b2e2c284e7bc1741c3df7a19b65977d1a6d1c4cbbc69ba44c4bfe165"} Jul 07 11:20:03.810505 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:03.810462 2585 generic.go:356] "Generic (PLEG): container finished" podID="09540c73-4f54-4415-99e5-06cbfc456237" containerID="66d3c2792fab582cfb1eedeb56d24b931943e8be8d13e4c4ef6888f7812522aa" exitCode=0 Jul 07 11:20:03.810964 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:03.810535 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" event={"ID":"09540c73-4f54-4415-99e5-06cbfc456237","Type":"ContainerDied","Data":"66d3c2792fab582cfb1eedeb56d24b931943e8be8d13e4c4ef6888f7812522aa"} Jul 07 11:20:04.964827 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:04.964793 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:20:05.090644 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.090566 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncl2b\" (UniqueName: \"kubernetes.io/projected/09540c73-4f54-4415-99e5-06cbfc456237-kube-api-access-ncl2b\") pod \"09540c73-4f54-4415-99e5-06cbfc456237\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " Jul 07 11:20:05.090644 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.090626 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-bundle\") pod \"09540c73-4f54-4415-99e5-06cbfc456237\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " Jul 07 11:20:05.090863 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.090661 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-util\") pod \"09540c73-4f54-4415-99e5-06cbfc456237\" (UID: \"09540c73-4f54-4415-99e5-06cbfc456237\") " Jul 07 11:20:05.091425 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.091394 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-bundle" (OuterVolumeSpecName: "bundle") pod "09540c73-4f54-4415-99e5-06cbfc456237" (UID: "09540c73-4f54-4415-99e5-06cbfc456237"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:20:05.092911 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.092883 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09540c73-4f54-4415-99e5-06cbfc456237-kube-api-access-ncl2b" (OuterVolumeSpecName: "kube-api-access-ncl2b") pod "09540c73-4f54-4415-99e5-06cbfc456237" (UID: "09540c73-4f54-4415-99e5-06cbfc456237"). InnerVolumeSpecName "kube-api-access-ncl2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:20:05.096035 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.096012 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-util" (OuterVolumeSpecName: "util") pod "09540c73-4f54-4415-99e5-06cbfc456237" (UID: "09540c73-4f54-4415-99e5-06cbfc456237"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:20:05.192008 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.191967 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-util\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:20:05.192008 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.192000 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncl2b\" (UniqueName: \"kubernetes.io/projected/09540c73-4f54-4415-99e5-06cbfc456237-kube-api-access-ncl2b\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:20:05.192008 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.192011 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09540c73-4f54-4415-99e5-06cbfc456237-bundle\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:20:05.818873 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.818843 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" Jul 07 11:20:05.818873 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.818850 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rmpp7" event={"ID":"09540c73-4f54-4415-99e5-06cbfc456237","Type":"ContainerDied","Data":"f8afd88ef74e281f23029ded72e002a539c7d74032c73d13eeb7694c500d7c02"} Jul 07 11:20:05.819073 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:05.818896 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8afd88ef74e281f23029ded72e002a539c7d74032c73d13eeb7694c500d7c02" Jul 07 11:20:10.551841 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.551806 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf"] Jul 07 11:20:10.552304 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.552086 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09540c73-4f54-4415-99e5-06cbfc456237" containerName="util" Jul 07 11:20:10.552304 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.552097 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="09540c73-4f54-4415-99e5-06cbfc456237" containerName="util" Jul 07 11:20:10.552304 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.552113 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09540c73-4f54-4415-99e5-06cbfc456237" containerName="extract" Jul 07 11:20:10.552304 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.552119 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="09540c73-4f54-4415-99e5-06cbfc456237" containerName="extract" Jul 07 11:20:10.552304 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.552127 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09540c73-4f54-4415-99e5-06cbfc456237" containerName="pull" Jul 07 11:20:10.552304 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.552132 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="09540c73-4f54-4415-99e5-06cbfc456237" containerName="pull" Jul 07 11:20:10.552304 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.552176 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="09540c73-4f54-4415-99e5-06cbfc456237" containerName="extract" Jul 07 11:20:10.555234 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.555217 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf" Jul 07 11:20:10.558103 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.558086 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 11:20:10.558384 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.558369 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ss6mc\"" Jul 07 11:20:10.558606 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.558592 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:20:10.567302 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.567283 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf"] Jul 07 11:20:10.633515 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.633486 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q89k5\" (UniqueName: \"kubernetes.io/projected/f65291fc-0dc3-4925-aa8f-89bbbb0ce58b-kube-api-access-q89k5\") pod \"cert-manager-operator-controller-manager-55c444bf86-lgpsf\" (UID: \"f65291fc-0dc3-4925-aa8f-89bbbb0ce58b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf" Jul 07 11:20:10.633686 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.633528 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f65291fc-0dc3-4925-aa8f-89bbbb0ce58b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lgpsf\" (UID: \"f65291fc-0dc3-4925-aa8f-89bbbb0ce58b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf" Jul 07 11:20:10.734405 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.734336 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q89k5\" (UniqueName: \"kubernetes.io/projected/f65291fc-0dc3-4925-aa8f-89bbbb0ce58b-kube-api-access-q89k5\") pod \"cert-manager-operator-controller-manager-55c444bf86-lgpsf\" (UID: \"f65291fc-0dc3-4925-aa8f-89bbbb0ce58b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf" Jul 07 11:20:10.734480 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.734431 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f65291fc-0dc3-4925-aa8f-89bbbb0ce58b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lgpsf\" (UID: \"f65291fc-0dc3-4925-aa8f-89bbbb0ce58b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf" Jul 07 11:20:10.734789 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.734774 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f65291fc-0dc3-4925-aa8f-89bbbb0ce58b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lgpsf\" (UID: \"f65291fc-0dc3-4925-aa8f-89bbbb0ce58b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf" Jul 07 11:20:10.743032 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.743002 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q89k5\" (UniqueName: \"kubernetes.io/projected/f65291fc-0dc3-4925-aa8f-89bbbb0ce58b-kube-api-access-q89k5\") pod \"cert-manager-operator-controller-manager-55c444bf86-lgpsf\" (UID: \"f65291fc-0dc3-4925-aa8f-89bbbb0ce58b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf" Jul 07 11:20:10.863164 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.863081 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf" Jul 07 11:20:10.995973 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:10.995949 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf"] Jul 07 11:20:10.999034 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:20:10.999004 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf65291fc_0dc3_4925_aa8f_89bbbb0ce58b.slice/crio-4b6fc8d9b307068ba63813ac155941e3ca8d2ab42542781f6efc28d255481aaa WatchSource:0}: Error finding container 4b6fc8d9b307068ba63813ac155941e3ca8d2ab42542781f6efc28d255481aaa: Status 404 returned error can't find the container with id 4b6fc8d9b307068ba63813ac155941e3ca8d2ab42542781f6efc28d255481aaa Jul 07 11:20:11.844751 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:11.844705 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf" event={"ID":"f65291fc-0dc3-4925-aa8f-89bbbb0ce58b","Type":"ContainerStarted","Data":"4b6fc8d9b307068ba63813ac155941e3ca8d2ab42542781f6efc28d255481aaa"} Jul 07 11:20:12.851733 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:12.851699 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf" event={"ID":"f65291fc-0dc3-4925-aa8f-89bbbb0ce58b","Type":"ContainerStarted","Data":"e684be96cc38093641357a7fe7fa82658840f53c064af1a81ed148f1113d4668"} Jul 07 11:20:12.884576 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:12.884466 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lgpsf" podStartSLOduration=1.252995889 podStartE2EDuration="2.884449895s" totalImagesPullingTime="1.631454006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:12.882838859 +0000 UTC m=+270.470705361" watchObservedRunningTime="2026-07-07 11:20:12.884449895 +0000 UTC m=+270.472316385" Jul 07 11:20:14.226591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.226554 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99"] Jul 07 11:20:14.229057 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.229033 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:14.231975 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.231955 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 11:20:14.232101 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.232028 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-n9mdv\"" Jul 07 11:20:14.232155 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.232115 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 11:20:14.241159 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.241137 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99"] Jul 07 11:20:14.259329 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.259295 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:14.259474 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.259340 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:14.259474 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.259408 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzdfb\" (UniqueName: \"kubernetes.io/projected/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-kube-api-access-tzdfb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:14.360060 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.360025 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:14.360210 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.360109 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:14.360210 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.360153 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzdfb\" (UniqueName: \"kubernetes.io/projected/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-kube-api-access-tzdfb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:14.360565 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.360542 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:14.360565 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.360558 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:14.393787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.393761 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzdfb\" (UniqueName: \"kubernetes.io/projected/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-kube-api-access-tzdfb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:14.538046 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.537958 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:14.621690 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.621662 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-lwtwm"] Jul 07 11:20:14.624134 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.624116 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" Jul 07 11:20:14.632152 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.632137 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 11:20:14.632422 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.632408 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 11:20:14.645118 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.645092 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-lwtwm"] Jul 07 11:20:14.661729 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.661696 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65r28\" (UniqueName: \"kubernetes.io/projected/4ab55754-b276-4599-89ff-e6103e351807-kube-api-access-65r28\") pod \"cert-manager-webhook-6486ffd997-lwtwm\" (UID: \"4ab55754-b276-4599-89ff-e6103e351807\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" Jul 07 11:20:14.661861 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.661779 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ab55754-b276-4599-89ff-e6103e351807-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-lwtwm\" (UID: \"4ab55754-b276-4599-89ff-e6103e351807\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" Jul 07 11:20:14.689381 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.689340 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99"] Jul 07 11:20:14.691699 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:20:14.691674 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd4b591_cf4c_4a9d_820e_d4b2d08e1476.slice/crio-1b532ff754c023e935541b3265b2cb9284d241e84f79130138933c4ad08fc22c WatchSource:0}: Error finding container 1b532ff754c023e935541b3265b2cb9284d241e84f79130138933c4ad08fc22c: Status 404 returned error can't find the container with id 1b532ff754c023e935541b3265b2cb9284d241e84f79130138933c4ad08fc22c Jul 07 11:20:14.762764 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.762736 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ab55754-b276-4599-89ff-e6103e351807-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-lwtwm\" (UID: \"4ab55754-b276-4599-89ff-e6103e351807\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" Jul 07 11:20:14.762908 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.762781 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65r28\" (UniqueName: \"kubernetes.io/projected/4ab55754-b276-4599-89ff-e6103e351807-kube-api-access-65r28\") pod \"cert-manager-webhook-6486ffd997-lwtwm\" (UID: \"4ab55754-b276-4599-89ff-e6103e351807\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" Jul 07 11:20:14.771717 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.771681 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ab55754-b276-4599-89ff-e6103e351807-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-lwtwm\" (UID: \"4ab55754-b276-4599-89ff-e6103e351807\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" Jul 07 11:20:14.772332 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.772313 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65r28\" (UniqueName: \"kubernetes.io/projected/4ab55754-b276-4599-89ff-e6103e351807-kube-api-access-65r28\") pod \"cert-manager-webhook-6486ffd997-lwtwm\" (UID: \"4ab55754-b276-4599-89ff-e6103e351807\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" Jul 07 11:20:14.861659 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.861579 2585 generic.go:356] "Generic (PLEG): container finished" podID="2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" containerID="d8e0214abd6a3a05429942a67121bbff9a11313c52f865517a458841ebe85fa7" exitCode=0 Jul 07 11:20:14.861792 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.861667 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" event={"ID":"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476","Type":"ContainerDied","Data":"d8e0214abd6a3a05429942a67121bbff9a11313c52f865517a458841ebe85fa7"} Jul 07 11:20:14.861792 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.861722 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" event={"ID":"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476","Type":"ContainerStarted","Data":"1b532ff754c023e935541b3265b2cb9284d241e84f79130138933c4ad08fc22c"} Jul 07 11:20:14.942875 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:14.942840 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" Jul 07 11:20:15.068662 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:15.068635 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-lwtwm"] Jul 07 11:20:15.070830 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:20:15.070802 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ab55754_b276_4599_89ff_e6103e351807.slice/crio-4655781d8340fab20860501f1e34b1623bbdf6603c229b200f6240542650c071 WatchSource:0}: Error finding container 4655781d8340fab20860501f1e34b1623bbdf6603c229b200f6240542650c071: Status 404 returned error can't find the container with id 4655781d8340fab20860501f1e34b1623bbdf6603c229b200f6240542650c071 Jul 07 11:20:15.868261 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:15.868202 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" event={"ID":"4ab55754-b276-4599-89ff-e6103e351807","Type":"ContainerStarted","Data":"4655781d8340fab20860501f1e34b1623bbdf6603c229b200f6240542650c071"} Jul 07 11:20:16.875074 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:16.875042 2585 generic.go:356] "Generic (PLEG): container finished" podID="2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" containerID="4c7aa05136a7a70f978e1589fd562657155358e801986d088901630dad8c476d" exitCode=0 Jul 07 11:20:16.875484 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:16.875128 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" event={"ID":"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476","Type":"ContainerDied","Data":"4c7aa05136a7a70f978e1589fd562657155358e801986d088901630dad8c476d"} Jul 07 11:20:17.347901 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.347811 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b88tx"] Jul 07 11:20:17.350137 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.350114 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b88tx" Jul 07 11:20:17.353684 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.353661 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8z48j\"" Jul 07 11:20:17.359269 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.359250 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b88tx"] Jul 07 11:20:17.385307 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.385269 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px4f7\" (UniqueName: \"kubernetes.io/projected/5a4ea214-cf78-4c6f-b7fc-5683873b79e2-kube-api-access-px4f7\") pod \"cert-manager-cainjector-559cdd8c87-b88tx\" (UID: \"5a4ea214-cf78-4c6f-b7fc-5683873b79e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b88tx" Jul 07 11:20:17.385475 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.385312 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a4ea214-cf78-4c6f-b7fc-5683873b79e2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b88tx\" (UID: \"5a4ea214-cf78-4c6f-b7fc-5683873b79e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b88tx" Jul 07 11:20:17.485832 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.485796 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px4f7\" (UniqueName: \"kubernetes.io/projected/5a4ea214-cf78-4c6f-b7fc-5683873b79e2-kube-api-access-px4f7\") pod \"cert-manager-cainjector-559cdd8c87-b88tx\" (UID: \"5a4ea214-cf78-4c6f-b7fc-5683873b79e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b88tx" Jul 07 11:20:17.485832 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.485835 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a4ea214-cf78-4c6f-b7fc-5683873b79e2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b88tx\" (UID: \"5a4ea214-cf78-4c6f-b7fc-5683873b79e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b88tx" Jul 07 11:20:17.494843 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.494803 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a4ea214-cf78-4c6f-b7fc-5683873b79e2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b88tx\" (UID: \"5a4ea214-cf78-4c6f-b7fc-5683873b79e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b88tx" Jul 07 11:20:17.495087 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.495069 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-px4f7\" (UniqueName: \"kubernetes.io/projected/5a4ea214-cf78-4c6f-b7fc-5683873b79e2-kube-api-access-px4f7\") pod \"cert-manager-cainjector-559cdd8c87-b88tx\" (UID: \"5a4ea214-cf78-4c6f-b7fc-5683873b79e2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b88tx" Jul 07 11:20:17.662122 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.662087 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b88tx" Jul 07 11:20:17.807981 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.807849 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b88tx"] Jul 07 11:20:17.882984 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.882948 2585 generic.go:356] "Generic (PLEG): container finished" podID="2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" containerID="d247e1e3eda65e32698ac7b76228fda0c20b7d47cdf431fdbd283a0eacef2a5f" exitCode=0 Jul 07 11:20:17.883403 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:17.882998 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" event={"ID":"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476","Type":"ContainerDied","Data":"d247e1e3eda65e32698ac7b76228fda0c20b7d47cdf431fdbd283a0eacef2a5f"} Jul 07 11:20:18.672206 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:20:18.672175 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a4ea214_cf78_4c6f_b7fc_5683873b79e2.slice/crio-7b24e5f9030a68c3a3a0e3906d0a828ef4333a1e7eb70f7197f906da44c86801 WatchSource:0}: Error finding container 7b24e5f9030a68c3a3a0e3906d0a828ef4333a1e7eb70f7197f906da44c86801: Status 404 returned error can't find the container with id 7b24e5f9030a68c3a3a0e3906d0a828ef4333a1e7eb70f7197f906da44c86801 Jul 07 11:20:18.888576 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:18.888538 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" event={"ID":"4ab55754-b276-4599-89ff-e6103e351807","Type":"ContainerStarted","Data":"43d37cb220fbec652a6e9ecb0f2deab1259dc1f1a3a4b5e8081c15c8ed4d365a"} Jul 07 11:20:18.889009 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:18.888590 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" Jul 07 11:20:18.889715 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:18.889690 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b88tx" event={"ID":"5a4ea214-cf78-4c6f-b7fc-5683873b79e2","Type":"ContainerStarted","Data":"7b24e5f9030a68c3a3a0e3906d0a828ef4333a1e7eb70f7197f906da44c86801"} Jul 07 11:20:18.905484 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:18.905430 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" podStartSLOduration=1.267773321 podStartE2EDuration="4.905408089s" totalImagesPullingTime="3.637634768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:18.905190829 +0000 UTC m=+276.493057318" watchObservedRunningTime="2026-07-07 11:20:18.905408089 +0000 UTC m=+276.493274582" Jul 07 11:20:19.012975 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.012952 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:19.100746 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.100714 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-bundle\") pod \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " Jul 07 11:20:19.100921 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.100756 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-util\") pod \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " Jul 07 11:20:19.100921 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.100840 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzdfb\" (UniqueName: \"kubernetes.io/projected/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-kube-api-access-tzdfb\") pod \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\" (UID: \"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476\") " Jul 07 11:20:19.101162 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.101143 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-bundle" (OuterVolumeSpecName: "bundle") pod "2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" (UID: "2dd4b591-cf4c-4a9d-820e-d4b2d08e1476"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:20:19.103024 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.103001 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-kube-api-access-tzdfb" (OuterVolumeSpecName: "kube-api-access-tzdfb") pod "2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" (UID: "2dd4b591-cf4c-4a9d-820e-d4b2d08e1476"). InnerVolumeSpecName "kube-api-access-tzdfb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:20:19.106312 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.106277 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-util" (OuterVolumeSpecName: "util") pod "2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" (UID: "2dd4b591-cf4c-4a9d-820e-d4b2d08e1476"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:20:19.201628 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.201541 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzdfb\" (UniqueName: \"kubernetes.io/projected/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-kube-api-access-tzdfb\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:20:19.201628 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.201571 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-bundle\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:20:19.201628 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.201580 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dd4b591-cf4c-4a9d-820e-d4b2d08e1476-util\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:20:19.896430 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.896402 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" Jul 07 11:20:19.896882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.896401 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgmg99" event={"ID":"2dd4b591-cf4c-4a9d-820e-d4b2d08e1476","Type":"ContainerDied","Data":"1b532ff754c023e935541b3265b2cb9284d241e84f79130138933c4ad08fc22c"} Jul 07 11:20:19.896882 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.896528 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b532ff754c023e935541b3265b2cb9284d241e84f79130138933c4ad08fc22c" Jul 07 11:20:19.897937 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.897907 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b88tx" event={"ID":"5a4ea214-cf78-4c6f-b7fc-5683873b79e2","Type":"ContainerStarted","Data":"b7cfe21905448eefa91067a211de8fba7fafea458f5b8a66e0e5add7e301f10d"} Jul 07 11:20:19.916543 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:19.916480 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b88tx" podStartSLOduration=2.323929603 podStartE2EDuration="2.916466037s" totalImagesPullingTime="592.536434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:19.915210316 +0000 UTC m=+277.503076805" watchObservedRunningTime="2026-07-07 11:20:19.916466037 +0000 UTC m=+277.504332527" Jul 07 11:20:24.900629 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:24.900600 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-lwtwm" Jul 07 11:20:27.436174 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.436140 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4"] Jul 07 11:20:27.436549 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.436431 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" containerName="extract" Jul 07 11:20:27.436549 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.436443 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" containerName="extract" Jul 07 11:20:27.436549 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.436451 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" containerName="pull" Jul 07 11:20:27.436549 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.436456 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" containerName="pull" Jul 07 11:20:27.436549 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.436465 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" containerName="util" Jul 07 11:20:27.436549 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.436471 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" containerName="util" Jul 07 11:20:27.436549 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.436528 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dd4b591-cf4c-4a9d-820e-d4b2d08e1476" containerName="extract" Jul 07 11:20:27.438847 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.438832 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4" Jul 07 11:20:27.441505 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.441481 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:20:27.441625 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.441528 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 11:20:27.443615 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.443596 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2phw4\"" Jul 07 11:20:27.448919 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.448898 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4"] Jul 07 11:20:27.476070 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.476035 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b23aeb58-bbe7-49da-87e8-69ef4cce1400-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pk2q4\" (UID: \"b23aeb58-bbe7-49da-87e8-69ef4cce1400\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4" Jul 07 11:20:27.476221 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.476101 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl8vp\" (UniqueName: \"kubernetes.io/projected/b23aeb58-bbe7-49da-87e8-69ef4cce1400-kube-api-access-dl8vp\") pod \"openshift-lws-operator-fd8ccff4c-pk2q4\" (UID: \"b23aeb58-bbe7-49da-87e8-69ef4cce1400\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4" Jul 07 11:20:27.577054 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.577016 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b23aeb58-bbe7-49da-87e8-69ef4cce1400-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pk2q4\" (UID: \"b23aeb58-bbe7-49da-87e8-69ef4cce1400\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4" Jul 07 11:20:27.577231 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.577077 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8vp\" (UniqueName: \"kubernetes.io/projected/b23aeb58-bbe7-49da-87e8-69ef4cce1400-kube-api-access-dl8vp\") pod \"openshift-lws-operator-fd8ccff4c-pk2q4\" (UID: \"b23aeb58-bbe7-49da-87e8-69ef4cce1400\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4" Jul 07 11:20:27.577440 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.577421 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b23aeb58-bbe7-49da-87e8-69ef4cce1400-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pk2q4\" (UID: \"b23aeb58-bbe7-49da-87e8-69ef4cce1400\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4" Jul 07 11:20:27.590960 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.590927 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl8vp\" (UniqueName: \"kubernetes.io/projected/b23aeb58-bbe7-49da-87e8-69ef4cce1400-kube-api-access-dl8vp\") pod \"openshift-lws-operator-fd8ccff4c-pk2q4\" (UID: \"b23aeb58-bbe7-49da-87e8-69ef4cce1400\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4" Jul 07 11:20:27.748530 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.748443 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4" Jul 07 11:20:27.901564 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.901522 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4"] Jul 07 11:20:27.903797 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:20:27.903774 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb23aeb58_bbe7_49da_87e8_69ef4cce1400.slice/crio-ca24e69783cf88306fa277a80d2df45176197a3d000cccf901868084a1a464cf WatchSource:0}: Error finding container ca24e69783cf88306fa277a80d2df45176197a3d000cccf901868084a1a464cf: Status 404 returned error can't find the container with id ca24e69783cf88306fa277a80d2df45176197a3d000cccf901868084a1a464cf Jul 07 11:20:27.932867 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:27.932837 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4" event={"ID":"b23aeb58-bbe7-49da-87e8-69ef4cce1400","Type":"ContainerStarted","Data":"ca24e69783cf88306fa277a80d2df45176197a3d000cccf901868084a1a464cf"} Jul 07 11:20:29.943795 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:29.943759 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4" event={"ID":"b23aeb58-bbe7-49da-87e8-69ef4cce1400","Type":"ContainerStarted","Data":"0265470ef5ada4a83cdca362beb0f340147160db71132b1c8bcbce5f7d89acad"} Jul 07 11:20:29.964918 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:29.964864 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pk2q4" podStartSLOduration=1.5704483 podStartE2EDuration="2.964847784s" totalImagesPullingTime="1.394399484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:29.964310523 +0000 UTC m=+287.552177015" watchObservedRunningTime="2026-07-07 11:20:29.964847784 +0000 UTC m=+287.552714273" Jul 07 11:20:32.708011 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.707975 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf"] Jul 07 11:20:32.710472 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.710455 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:32.713269 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.713250 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 11:20:32.714483 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.714468 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 11:20:32.714539 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.714513 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-n9mdv\"" Jul 07 11:20:32.721204 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.721174 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf"] Jul 07 11:20:32.826421 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.826389 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:32.826585 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.826425 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6txrb\" (UniqueName: \"kubernetes.io/projected/31aab5ce-ddfa-459e-8e76-4684d6539f5e-kube-api-access-6txrb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:32.826585 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.826466 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:32.926899 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.926868 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:32.927034 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.926933 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:32.927034 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.926966 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6txrb\" (UniqueName: \"kubernetes.io/projected/31aab5ce-ddfa-459e-8e76-4684d6539f5e-kube-api-access-6txrb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:32.927272 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.927255 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:32.927325 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.927305 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:32.935332 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:32.935306 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6txrb\" (UniqueName: \"kubernetes.io/projected/31aab5ce-ddfa-459e-8e76-4684d6539f5e-kube-api-access-6txrb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:33.019263 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:33.019170 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:33.148047 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:33.148020 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf"] Jul 07 11:20:33.150625 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:20:33.150598 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31aab5ce_ddfa_459e_8e76_4684d6539f5e.slice/crio-28c950b3215258fb418801fc45a4a957bd9c418024b9d06887bc8a1829c9b002 WatchSource:0}: Error finding container 28c950b3215258fb418801fc45a4a957bd9c418024b9d06887bc8a1829c9b002: Status 404 returned error can't find the container with id 28c950b3215258fb418801fc45a4a957bd9c418024b9d06887bc8a1829c9b002 Jul 07 11:20:33.962181 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:33.962147 2585 generic.go:356] "Generic (PLEG): container finished" podID="31aab5ce-ddfa-459e-8e76-4684d6539f5e" containerID="c54c4db8309e941832f8b85cef5fbedafe04430b9710ff8c27829af212d390a5" exitCode=0 Jul 07 11:20:33.962564 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:33.962234 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" event={"ID":"31aab5ce-ddfa-459e-8e76-4684d6539f5e","Type":"ContainerDied","Data":"c54c4db8309e941832f8b85cef5fbedafe04430b9710ff8c27829af212d390a5"} Jul 07 11:20:33.962564 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:33.962283 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" event={"ID":"31aab5ce-ddfa-459e-8e76-4684d6539f5e","Type":"ContainerStarted","Data":"28c950b3215258fb418801fc45a4a957bd9c418024b9d06887bc8a1829c9b002"} Jul 07 11:20:35.976136 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:35.976096 2585 generic.go:356] "Generic (PLEG): container finished" podID="31aab5ce-ddfa-459e-8e76-4684d6539f5e" containerID="2d5137112d34c1a4a91e03660dccc3571f92ac1de52bbec48d36c84893d304b2" exitCode=0 Jul 07 11:20:35.976719 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:35.976141 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" event={"ID":"31aab5ce-ddfa-459e-8e76-4684d6539f5e","Type":"ContainerDied","Data":"2d5137112d34c1a4a91e03660dccc3571f92ac1de52bbec48d36c84893d304b2"} Jul 07 11:20:36.982652 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:36.982620 2585 generic.go:356] "Generic (PLEG): container finished" podID="31aab5ce-ddfa-459e-8e76-4684d6539f5e" containerID="ca97aea95d245d5ae03fefb49985e9072297485bcb020c23bed866b5c3abe41e" exitCode=0 Jul 07 11:20:36.983020 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:36.982688 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" event={"ID":"31aab5ce-ddfa-459e-8e76-4684d6539f5e","Type":"ContainerDied","Data":"ca97aea95d245d5ae03fefb49985e9072297485bcb020c23bed866b5c3abe41e"} Jul 07 11:20:38.114873 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.114848 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:38.172737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.172697 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6txrb\" (UniqueName: \"kubernetes.io/projected/31aab5ce-ddfa-459e-8e76-4684d6539f5e-kube-api-access-6txrb\") pod \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " Jul 07 11:20:38.172913 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.172777 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-util\") pod \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " Jul 07 11:20:38.172970 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.172920 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-bundle\") pod \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\" (UID: \"31aab5ce-ddfa-459e-8e76-4684d6539f5e\") " Jul 07 11:20:38.173570 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.173544 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-bundle" (OuterVolumeSpecName: "bundle") pod "31aab5ce-ddfa-459e-8e76-4684d6539f5e" (UID: "31aab5ce-ddfa-459e-8e76-4684d6539f5e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:20:38.175055 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.175032 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31aab5ce-ddfa-459e-8e76-4684d6539f5e-kube-api-access-6txrb" (OuterVolumeSpecName: "kube-api-access-6txrb") pod "31aab5ce-ddfa-459e-8e76-4684d6539f5e" (UID: "31aab5ce-ddfa-459e-8e76-4684d6539f5e"). InnerVolumeSpecName "kube-api-access-6txrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:20:38.178251 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.178218 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-util" (OuterVolumeSpecName: "util") pod "31aab5ce-ddfa-459e-8e76-4684d6539f5e" (UID: "31aab5ce-ddfa-459e-8e76-4684d6539f5e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:20:38.274528 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.274444 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6txrb\" (UniqueName: \"kubernetes.io/projected/31aab5ce-ddfa-459e-8e76-4684d6539f5e-kube-api-access-6txrb\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:20:38.274528 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.274472 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-util\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:20:38.274528 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.274481 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31aab5ce-ddfa-459e-8e76-4684d6539f5e-bundle\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:20:38.993229 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.993194 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" event={"ID":"31aab5ce-ddfa-459e-8e76-4684d6539f5e","Type":"ContainerDied","Data":"28c950b3215258fb418801fc45a4a957bd9c418024b9d06887bc8a1829c9b002"} Jul 07 11:20:38.993229 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.993213 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvzhhf" Jul 07 11:20:38.993467 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:38.993241 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28c950b3215258fb418801fc45a4a957bd9c418024b9d06887bc8a1829c9b002" Jul 07 11:20:43.478810 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:43.478781 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:20:43.480192 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:43.480168 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:20:43.486458 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:43.486438 2585 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 11:20:51.201605 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.201571 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx"] Jul 07 11:20:51.205898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.201995 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31aab5ce-ddfa-459e-8e76-4684d6539f5e" containerName="extract" Jul 07 11:20:51.205898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.202012 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="31aab5ce-ddfa-459e-8e76-4684d6539f5e" containerName="extract" Jul 07 11:20:51.205898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.202028 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31aab5ce-ddfa-459e-8e76-4684d6539f5e" containerName="util" Jul 07 11:20:51.205898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.202034 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="31aab5ce-ddfa-459e-8e76-4684d6539f5e" containerName="util" Jul 07 11:20:51.205898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.202051 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31aab5ce-ddfa-459e-8e76-4684d6539f5e" containerName="pull" Jul 07 11:20:51.205898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.202057 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="31aab5ce-ddfa-459e-8e76-4684d6539f5e" containerName="pull" Jul 07 11:20:51.205898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.202111 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="31aab5ce-ddfa-459e-8e76-4684d6539f5e" containerName="extract" Jul 07 11:20:51.206333 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.205916 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.210249 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.210222 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 11:20:51.210388 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.210284 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 11:20:51.210583 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.210565 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 11:20:51.212523 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.212501 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tp44d\"" Jul 07 11:20:51.213902 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.213886 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 11:20:51.221022 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.221002 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx"] Jul 07 11:20:51.390495 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.390450 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9887f11-40f8-4adc-9cf4-e3df0ce77560-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.390495 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.390495 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg7gs\" (UniqueName: \"kubernetes.io/projected/d9887f11-40f8-4adc-9cf4-e3df0ce77560-kube-api-access-lg7gs\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.390740 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.390566 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9887f11-40f8-4adc-9cf4-e3df0ce77560-manifests\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.390740 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.390611 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9887f11-40f8-4adc-9cf4-e3df0ce77560-tmp\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.390740 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.390646 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9887f11-40f8-4adc-9cf4-e3df0ce77560-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.491812 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.491718 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9887f11-40f8-4adc-9cf4-e3df0ce77560-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.491812 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.491754 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg7gs\" (UniqueName: \"kubernetes.io/projected/d9887f11-40f8-4adc-9cf4-e3df0ce77560-kube-api-access-lg7gs\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.491812 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.491787 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9887f11-40f8-4adc-9cf4-e3df0ce77560-manifests\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.491812 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.491811 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9887f11-40f8-4adc-9cf4-e3df0ce77560-tmp\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.492144 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.491831 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9887f11-40f8-4adc-9cf4-e3df0ce77560-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.492307 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.492286 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d9887f11-40f8-4adc-9cf4-e3df0ce77560-manifests\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.494082 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.494051 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9887f11-40f8-4adc-9cf4-e3df0ce77560-tmp\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.494381 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.494332 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9887f11-40f8-4adc-9cf4-e3df0ce77560-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.494462 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.494369 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9887f11-40f8-4adc-9cf4-e3df0ce77560-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.503594 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.503575 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg7gs\" (UniqueName: \"kubernetes.io/projected/d9887f11-40f8-4adc-9cf4-e3df0ce77560-kube-api-access-lg7gs\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-99mxx\" (UID: \"d9887f11-40f8-4adc-9cf4-e3df0ce77560\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.516484 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.516463 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:51.660648 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.660626 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx"] Jul 07 11:20:51.661964 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:20:51.661943 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9887f11_40f8_4adc_9cf4_e3df0ce77560.slice/crio-284a89d563810a821806e048533efcb259e4fedd86e93bd4830db31e4581820d WatchSource:0}: Error finding container 284a89d563810a821806e048533efcb259e4fedd86e93bd4830db31e4581820d: Status 404 returned error can't find the container with id 284a89d563810a821806e048533efcb259e4fedd86e93bd4830db31e4581820d Jul 07 11:20:51.663839 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:51.663821 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:20:52.051419 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:52.051378 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" event={"ID":"d9887f11-40f8-4adc-9cf4-e3df0ce77560","Type":"ContainerStarted","Data":"284a89d563810a821806e048533efcb259e4fedd86e93bd4830db31e4581820d"} Jul 07 11:20:55.067910 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:55.067873 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" event={"ID":"d9887f11-40f8-4adc-9cf4-e3df0ce77560","Type":"ContainerStarted","Data":"49b082d966e2e3febb45dae587e804d37b6bdcd0769e700df2aa5e0822a9e62f"} Jul 07 11:20:56.074200 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:56.074113 2585 generic.go:356] "Generic (PLEG): container finished" podID="d9887f11-40f8-4adc-9cf4-e3df0ce77560" containerID="49b082d966e2e3febb45dae587e804d37b6bdcd0769e700df2aa5e0822a9e62f" exitCode=0 Jul 07 11:20:56.074200 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:56.074149 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" event={"ID":"d9887f11-40f8-4adc-9cf4-e3df0ce77560","Type":"ContainerDied","Data":"49b082d966e2e3febb45dae587e804d37b6bdcd0769e700df2aa5e0822a9e62f"} Jul 07 11:20:57.081900 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:57.081867 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" event={"ID":"d9887f11-40f8-4adc-9cf4-e3df0ce77560","Type":"ContainerStarted","Data":"3304851b619c091d8882f3e1b1555ce96787a0a3a1c4f7452608b4a9ccc941f3"} Jul 07 11:20:57.082390 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:57.081984 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:20:57.123008 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:20:57.122957 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" podStartSLOduration=1.8648905789999999 podStartE2EDuration="6.122941059s" totalImagesPullingTime="3.241256012s" totalInitContainerRuntime="1.016794468s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:57.121093321 +0000 UTC m=+314.708959811" watchObservedRunningTime="2026-07-07 11:20:57.122941059 +0000 UTC m=+314.710807548" Jul 07 11:21:08.088708 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:08.088674 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-99mxx" Jul 07 11:21:10.886429 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:10.886394 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-785759898c-qqwxb"] Jul 07 11:21:10.889067 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:10.889045 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" Jul 07 11:21:10.895994 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:10.895388 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 11:21:10.895994 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:10.895406 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 11:21:10.895994 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:10.895745 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hq5xl\"" Jul 07 11:21:10.895994 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:10.895761 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 11:21:10.895994 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:10.895994 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 11:21:10.915066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:10.915040 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-785759898c-qqwxb"] Jul 07 11:21:10.953339 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:10.953301 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67nln\" (UniqueName: \"kubernetes.io/projected/05c36606-981a-422a-8c40-4cb33f3e547f-kube-api-access-67nln\") pod \"kube-auth-proxy-785759898c-qqwxb\" (UID: \"05c36606-981a-422a-8c40-4cb33f3e547f\") " pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" Jul 07 11:21:10.953529 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:10.953472 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05c36606-981a-422a-8c40-4cb33f3e547f-tmp\") pod \"kube-auth-proxy-785759898c-qqwxb\" (UID: \"05c36606-981a-422a-8c40-4cb33f3e547f\") " pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" Jul 07 11:21:10.953529 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:10.953525 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05c36606-981a-422a-8c40-4cb33f3e547f-tls-certs\") pod \"kube-auth-proxy-785759898c-qqwxb\" (UID: \"05c36606-981a-422a-8c40-4cb33f3e547f\") " pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" Jul 07 11:21:11.032571 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.032538 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4vwpx"] Jul 07 11:21:11.034728 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.034704 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" Jul 07 11:21:11.037584 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.037562 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dbcn5\"" Jul 07 11:21:11.037791 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.037777 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 07 11:21:11.049760 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.049736 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4vwpx"] Jul 07 11:21:11.054672 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.054647 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67nln\" (UniqueName: \"kubernetes.io/projected/05c36606-981a-422a-8c40-4cb33f3e547f-kube-api-access-67nln\") pod \"kube-auth-proxy-785759898c-qqwxb\" (UID: \"05c36606-981a-422a-8c40-4cb33f3e547f\") " pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" Jul 07 11:21:11.054850 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.054685 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fqp5\" (UniqueName: \"kubernetes.io/projected/5dbcfd76-bbd9-44f0-98e2-b9bfac799f77-kube-api-access-2fqp5\") pod \"odh-model-controller-7998c67dc4-4vwpx\" (UID: \"5dbcfd76-bbd9-44f0-98e2-b9bfac799f77\") " pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" Jul 07 11:21:11.054850 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.054763 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05c36606-981a-422a-8c40-4cb33f3e547f-tmp\") pod \"kube-auth-proxy-785759898c-qqwxb\" (UID: \"05c36606-981a-422a-8c40-4cb33f3e547f\") " pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" Jul 07 11:21:11.054850 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.054799 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05c36606-981a-422a-8c40-4cb33f3e547f-tls-certs\") pod \"kube-auth-proxy-785759898c-qqwxb\" (UID: \"05c36606-981a-422a-8c40-4cb33f3e547f\") " pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" Jul 07 11:21:11.054850 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.054824 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dbcfd76-bbd9-44f0-98e2-b9bfac799f77-cert\") pod \"odh-model-controller-7998c67dc4-4vwpx\" (UID: \"5dbcfd76-bbd9-44f0-98e2-b9bfac799f77\") " pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" Jul 07 11:21:11.057086 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.057058 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05c36606-981a-422a-8c40-4cb33f3e547f-tmp\") pod \"kube-auth-proxy-785759898c-qqwxb\" (UID: \"05c36606-981a-422a-8c40-4cb33f3e547f\") " pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" Jul 07 11:21:11.057358 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.057326 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05c36606-981a-422a-8c40-4cb33f3e547f-tls-certs\") pod \"kube-auth-proxy-785759898c-qqwxb\" (UID: \"05c36606-981a-422a-8c40-4cb33f3e547f\") " pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" Jul 07 11:21:11.079318 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.079288 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67nln\" (UniqueName: \"kubernetes.io/projected/05c36606-981a-422a-8c40-4cb33f3e547f-kube-api-access-67nln\") pod \"kube-auth-proxy-785759898c-qqwxb\" (UID: \"05c36606-981a-422a-8c40-4cb33f3e547f\") " pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" Jul 07 11:21:11.155666 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.155574 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fqp5\" (UniqueName: \"kubernetes.io/projected/5dbcfd76-bbd9-44f0-98e2-b9bfac799f77-kube-api-access-2fqp5\") pod \"odh-model-controller-7998c67dc4-4vwpx\" (UID: \"5dbcfd76-bbd9-44f0-98e2-b9bfac799f77\") " pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" Jul 07 11:21:11.155666 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.155663 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dbcfd76-bbd9-44f0-98e2-b9bfac799f77-cert\") pod \"odh-model-controller-7998c67dc4-4vwpx\" (UID: \"5dbcfd76-bbd9-44f0-98e2-b9bfac799f77\") " pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" Jul 07 11:21:11.155884 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:21:11.155782 2585 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 07 11:21:11.155884 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:21:11.155848 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dbcfd76-bbd9-44f0-98e2-b9bfac799f77-cert podName:5dbcfd76-bbd9-44f0-98e2-b9bfac799f77 nodeName:}" failed. No retries permitted until 2026-07-07 11:21:11.655826262 +0000 UTC m=+329.243692745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5dbcfd76-bbd9-44f0-98e2-b9bfac799f77-cert") pod "odh-model-controller-7998c67dc4-4vwpx" (UID: "5dbcfd76-bbd9-44f0-98e2-b9bfac799f77") : secret "odh-model-controller-webhook-cert" not found Jul 07 11:21:11.168183 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.168154 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fqp5\" (UniqueName: \"kubernetes.io/projected/5dbcfd76-bbd9-44f0-98e2-b9bfac799f77-kube-api-access-2fqp5\") pod \"odh-model-controller-7998c67dc4-4vwpx\" (UID: \"5dbcfd76-bbd9-44f0-98e2-b9bfac799f77\") " pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" Jul 07 11:21:11.198992 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.198958 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" Jul 07 11:21:11.345847 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.345823 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-785759898c-qqwxb"] Jul 07 11:21:11.348237 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:21:11.348211 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05c36606_981a_422a_8c40_4cb33f3e547f.slice/crio-47309247c7cf07ada31720c8f8b598ac57043f1742ca6dfb31a3c52f89759709 WatchSource:0}: Error finding container 47309247c7cf07ada31720c8f8b598ac57043f1742ca6dfb31a3c52f89759709: Status 404 returned error can't find the container with id 47309247c7cf07ada31720c8f8b598ac57043f1742ca6dfb31a3c52f89759709 Jul 07 11:21:11.660653 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.660616 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dbcfd76-bbd9-44f0-98e2-b9bfac799f77-cert\") pod \"odh-model-controller-7998c67dc4-4vwpx\" (UID: \"5dbcfd76-bbd9-44f0-98e2-b9bfac799f77\") " pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" Jul 07 11:21:11.662960 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.662939 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dbcfd76-bbd9-44f0-98e2-b9bfac799f77-cert\") pod \"odh-model-controller-7998c67dc4-4vwpx\" (UID: \"5dbcfd76-bbd9-44f0-98e2-b9bfac799f77\") " pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" Jul 07 11:21:11.945685 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:11.945607 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" Jul 07 11:21:12.045110 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.045078 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ld52x"] Jul 07 11:21:12.048463 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.048437 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" Jul 07 11:21:12.051442 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.051407 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 07 11:21:12.051751 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.051732 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-sr8nt\"" Jul 07 11:21:12.066980 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.064920 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ld52x"] Jul 07 11:21:12.121425 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.121401 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-4vwpx"] Jul 07 11:21:12.124330 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:21:12.124299 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dbcfd76_bbd9_44f0_98e2_b9bfac799f77.slice/crio-396301ae180d2bf495df985fce9374214437a424a60af4d5149c5a83d07050e9 WatchSource:0}: Error finding container 396301ae180d2bf495df985fce9374214437a424a60af4d5149c5a83d07050e9: Status 404 returned error can't find the container with id 396301ae180d2bf495df985fce9374214437a424a60af4d5149c5a83d07050e9 Jul 07 11:21:12.166261 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.166229 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd2c4e09-6656-4c98-a5cc-3451a3b01528-tls-certs\") pod \"model-serving-api-54d89bd5fc-ld52x\" (UID: \"fd2c4e09-6656-4c98-a5cc-3451a3b01528\") " pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" Jul 07 11:21:12.166541 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.166521 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6mtf\" (UniqueName: \"kubernetes.io/projected/fd2c4e09-6656-4c98-a5cc-3451a3b01528-kube-api-access-c6mtf\") pod \"model-serving-api-54d89bd5fc-ld52x\" (UID: \"fd2c4e09-6656-4c98-a5cc-3451a3b01528\") " pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" Jul 07 11:21:12.167486 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.167451 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" event={"ID":"5dbcfd76-bbd9-44f0-98e2-b9bfac799f77","Type":"ContainerStarted","Data":"396301ae180d2bf495df985fce9374214437a424a60af4d5149c5a83d07050e9"} Jul 07 11:21:12.168928 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.168897 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" event={"ID":"05c36606-981a-422a-8c40-4cb33f3e547f","Type":"ContainerStarted","Data":"47309247c7cf07ada31720c8f8b598ac57043f1742ca6dfb31a3c52f89759709"} Jul 07 11:21:12.267492 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.267378 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd2c4e09-6656-4c98-a5cc-3451a3b01528-tls-certs\") pod \"model-serving-api-54d89bd5fc-ld52x\" (UID: \"fd2c4e09-6656-4c98-a5cc-3451a3b01528\") " pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" Jul 07 11:21:12.267679 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.267559 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6mtf\" (UniqueName: \"kubernetes.io/projected/fd2c4e09-6656-4c98-a5cc-3451a3b01528-kube-api-access-c6mtf\") pod \"model-serving-api-54d89bd5fc-ld52x\" (UID: \"fd2c4e09-6656-4c98-a5cc-3451a3b01528\") " pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" Jul 07 11:21:12.270649 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.270607 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd2c4e09-6656-4c98-a5cc-3451a3b01528-tls-certs\") pod \"model-serving-api-54d89bd5fc-ld52x\" (UID: \"fd2c4e09-6656-4c98-a5cc-3451a3b01528\") " pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" Jul 07 11:21:12.281737 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.281607 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6mtf\" (UniqueName: \"kubernetes.io/projected/fd2c4e09-6656-4c98-a5cc-3451a3b01528-kube-api-access-c6mtf\") pod \"model-serving-api-54d89bd5fc-ld52x\" (UID: \"fd2c4e09-6656-4c98-a5cc-3451a3b01528\") " pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" Jul 07 11:21:12.383705 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.383669 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" Jul 07 11:21:12.413109 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.413074 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv"] Jul 07 11:21:12.415525 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.415500 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.418762 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.418661 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 11:21:12.418762 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.418678 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 11:21:12.419148 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.419127 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 11:21:12.419247 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.419127 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 11:21:12.419247 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.419212 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pb7sv\"" Jul 07 11:21:12.442283 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.442217 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv"] Jul 07 11:21:12.470061 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.469797 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.470061 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.469848 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-cacerts\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.470061 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.469900 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-token\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.470061 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.469938 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.470061 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.469968 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-local-certs\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.470493 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.470081 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.470493 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.470113 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44hmm\" (UniqueName: \"kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-kube-api-access-44hmm\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.537426 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.537400 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ld52x"] Jul 07 11:21:12.539072 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:21:12.539046 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2c4e09_6656_4c98_a5cc_3451a3b01528.slice/crio-b4be71731f5bc29070e5251865f431afbd4c3fad6ba36d88ac66d88dac45e2ff WatchSource:0}: Error finding container b4be71731f5bc29070e5251865f431afbd4c3fad6ba36d88ac66d88dac45e2ff: Status 404 returned error can't find the container with id b4be71731f5bc29070e5251865f431afbd4c3fad6ba36d88ac66d88dac45e2ff Jul 07 11:21:12.570883 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.570850 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.571071 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.570897 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44hmm\" (UniqueName: \"kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-kube-api-access-44hmm\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.571071 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.570948 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.571071 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.570975 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-cacerts\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.571071 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.571021 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-token\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.571071 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.571053 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.571582 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.571086 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-local-certs\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.571736 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.571678 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.574052 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.574027 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-local-certs\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.574207 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.574190 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.574977 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.574951 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.575093 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.574989 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-cacerts\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.584538 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.584476 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-token\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.584898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.584878 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44hmm\" (UniqueName: \"kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-kube-api-access-44hmm\") pod \"istiod-openshift-gateway-7574459656-2dwcv\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.726594 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.726554 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:12.883553 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:12.883527 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv"] Jul 07 11:21:12.887159 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:21:12.886724 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1a2da85_c2de_4e2a_9b66_36755b69e8b1.slice/crio-245ded488de5d05e9804b7214069f65d588031753ec9c11da8cd3b17422d613c WatchSource:0}: Error finding container 245ded488de5d05e9804b7214069f65d588031753ec9c11da8cd3b17422d613c: Status 404 returned error can't find the container with id 245ded488de5d05e9804b7214069f65d588031753ec9c11da8cd3b17422d613c Jul 07 11:21:13.180236 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:13.180187 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" event={"ID":"fd2c4e09-6656-4c98-a5cc-3451a3b01528","Type":"ContainerStarted","Data":"b4be71731f5bc29070e5251865f431afbd4c3fad6ba36d88ac66d88dac45e2ff"} Jul 07 11:21:13.182339 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:13.182305 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" event={"ID":"c1a2da85-c2de-4e2a-9b66-36755b69e8b1","Type":"ContainerStarted","Data":"245ded488de5d05e9804b7214069f65d588031753ec9c11da8cd3b17422d613c"} Jul 07 11:21:17.445874 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:17.445833 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 11:21:17.446168 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:17.445918 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 11:21:18.213085 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.213041 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" event={"ID":"05c36606-981a-422a-8c40-4cb33f3e547f","Type":"ContainerStarted","Data":"81510505d3101062b8f494ecd1b1a142929a60bfa1da3681a196d254ce1ad898"} Jul 07 11:21:18.214677 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.214641 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" event={"ID":"c1a2da85-c2de-4e2a-9b66-36755b69e8b1","Type":"ContainerStarted","Data":"741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9"} Jul 07 11:21:18.214829 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.214703 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:18.216229 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.216205 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" event={"ID":"5dbcfd76-bbd9-44f0-98e2-b9bfac799f77","Type":"ContainerStarted","Data":"94c4387b95e6e478ad2f1d06a35e9dde08f30015bd986b803d4f6090a6e55e54"} Jul 07 11:21:18.216372 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.216271 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" Jul 07 11:21:18.216578 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.216543 2585 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-2dwcv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 11:21:18.216774 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.216605 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" podUID="c1a2da85-c2de-4e2a-9b66-36755b69e8b1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 11:21:18.217823 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.217804 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" event={"ID":"fd2c4e09-6656-4c98-a5cc-3451a3b01528","Type":"ContainerStarted","Data":"7aae7d5242f567272c7da9d1b5f4db20e12eac24cf2fd56274269a2804073a5c"} Jul 07 11:21:18.217945 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.217930 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" Jul 07 11:21:18.239852 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.239795 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-785759898c-qqwxb" podStartSLOduration=2.161864138 podStartE2EDuration="8.239777626s" totalImagesPullingTime="6.077913488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:18.238667177 +0000 UTC m=+335.826533667" watchObservedRunningTime="2026-07-07 11:21:18.239777626 +0000 UTC m=+335.827644161" Jul 07 11:21:18.258863 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.258809 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" podStartSLOduration=1.368327909 podStartE2EDuration="6.258778691s" totalImagesPullingTime="4.890450782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:18.257435556 +0000 UTC m=+335.845302048" watchObservedRunningTime="2026-07-07 11:21:18.258778691 +0000 UTC m=+335.846645173" Jul 07 11:21:18.283311 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.283141 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" podStartSLOduration=1.96732894 podStartE2EDuration="7.283120903s" totalImagesPullingTime="5.315791963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:18.279770083 +0000 UTC m=+335.867636574" watchObservedRunningTime="2026-07-07 11:21:18.283120903 +0000 UTC m=+335.870987394" Jul 07 11:21:18.305033 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:18.304964 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" podStartSLOduration=1.749335466 podStartE2EDuration="6.304943252s" totalImagesPullingTime="4.555607786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:18.304309337 +0000 UTC m=+335.892175828" watchObservedRunningTime="2026-07-07 11:21:18.304943252 +0000 UTC m=+335.892809743" Jul 07 11:21:19.223238 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:19.223201 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:23.023556 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.023516 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4r68v"] Jul 07 11:21:23.026708 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.026693 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" Jul 07 11:21:23.031664 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.031641 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 11:21:23.032411 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.032395 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-52694\"" Jul 07 11:21:23.055596 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.055559 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz"] Jul 07 11:21:23.059524 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.059500 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4r68v"] Jul 07 11:21:23.059665 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.059592 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:21:23.063238 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.063217 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-f8nzv\"" Jul 07 11:21:23.067323 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.067305 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 11:21:23.076754 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.076728 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cf720f3-0e4c-4731-821c-087de85bacf8-cert\") pod \"kserve-controller-manager-d6f4b65b8-4r68v\" (UID: \"5cf720f3-0e4c-4731-821c-087de85bacf8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" Jul 07 11:21:23.076889 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.076767 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9rfh\" (UniqueName: \"kubernetes.io/projected/5cf720f3-0e4c-4731-821c-087de85bacf8-kube-api-access-f9rfh\") pod \"kserve-controller-manager-d6f4b65b8-4r68v\" (UID: \"5cf720f3-0e4c-4731-821c-087de85bacf8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" Jul 07 11:21:23.087446 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.087418 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz"] Jul 07 11:21:23.177256 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.177217 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cf720f3-0e4c-4731-821c-087de85bacf8-cert\") pod \"kserve-controller-manager-d6f4b65b8-4r68v\" (UID: \"5cf720f3-0e4c-4731-821c-087de85bacf8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" Jul 07 11:21:23.177256 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.177263 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf9wb\" (UniqueName: \"kubernetes.io/projected/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-kube-api-access-hf9wb\") pod \"llmisvc-controller-manager-777c6bd7d5-v5gtz\" (UID: \"0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:21:23.177532 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.177285 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-v5gtz\" (UID: \"0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:21:23.177532 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.177303 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9rfh\" (UniqueName: \"kubernetes.io/projected/5cf720f3-0e4c-4731-821c-087de85bacf8-kube-api-access-f9rfh\") pod \"kserve-controller-manager-d6f4b65b8-4r68v\" (UID: \"5cf720f3-0e4c-4731-821c-087de85bacf8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" Jul 07 11:21:23.177532 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:21:23.177407 2585 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 07 11:21:23.177532 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:21:23.177475 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf720f3-0e4c-4731-821c-087de85bacf8-cert podName:5cf720f3-0e4c-4731-821c-087de85bacf8 nodeName:}" failed. No retries permitted until 2026-07-07 11:21:23.677456237 +0000 UTC m=+341.265322708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5cf720f3-0e4c-4731-821c-087de85bacf8-cert") pod "kserve-controller-manager-d6f4b65b8-4r68v" (UID: "5cf720f3-0e4c-4731-821c-087de85bacf8") : secret "kserve-webhook-server-cert" not found Jul 07 11:21:23.192246 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.192219 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9rfh\" (UniqueName: \"kubernetes.io/projected/5cf720f3-0e4c-4731-821c-087de85bacf8-kube-api-access-f9rfh\") pod \"kserve-controller-manager-d6f4b65b8-4r68v\" (UID: \"5cf720f3-0e4c-4731-821c-087de85bacf8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" Jul 07 11:21:23.278575 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.278480 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf9wb\" (UniqueName: \"kubernetes.io/projected/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-kube-api-access-hf9wb\") pod \"llmisvc-controller-manager-777c6bd7d5-v5gtz\" (UID: \"0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:21:23.278575 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.278518 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-v5gtz\" (UID: \"0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:21:23.278822 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:21:23.278634 2585 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 07 11:21:23.278822 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:21:23.278696 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-cert podName:0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb nodeName:}" failed. No retries permitted until 2026-07-07 11:21:23.778681453 +0000 UTC m=+341.366547922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-cert") pod "llmisvc-controller-manager-777c6bd7d5-v5gtz" (UID: "0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb") : secret "llmisvc-webhook-server-cert" not found Jul 07 11:21:23.293208 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.293175 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf9wb\" (UniqueName: \"kubernetes.io/projected/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-kube-api-access-hf9wb\") pod \"llmisvc-controller-manager-777c6bd7d5-v5gtz\" (UID: \"0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:21:23.682057 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.682026 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cf720f3-0e4c-4731-821c-087de85bacf8-cert\") pod \"kserve-controller-manager-d6f4b65b8-4r68v\" (UID: \"5cf720f3-0e4c-4731-821c-087de85bacf8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" Jul 07 11:21:23.684528 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.684503 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cf720f3-0e4c-4731-821c-087de85bacf8-cert\") pod \"kserve-controller-manager-d6f4b65b8-4r68v\" (UID: \"5cf720f3-0e4c-4731-821c-087de85bacf8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" Jul 07 11:21:23.783412 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.783373 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-v5gtz\" (UID: \"0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:21:23.783570 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:21:23.783538 2585 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 07 11:21:23.783612 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:21:23.783607 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-cert podName:0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb nodeName:}" failed. No retries permitted until 2026-07-07 11:21:24.783591551 +0000 UTC m=+342.371458023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-cert") pod "llmisvc-controller-manager-777c6bd7d5-v5gtz" (UID: "0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb") : secret "llmisvc-webhook-server-cert" not found Jul 07 11:21:23.936708 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:23.936617 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" Jul 07 11:21:24.100736 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:24.100700 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4r68v"] Jul 07 11:21:24.104547 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:21:24.104522 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cf720f3_0e4c_4731_821c_087de85bacf8.slice/crio-630f2c2b125e02922b55679b9483bad40bb7affb105b221fffbe651cd1528d15 WatchSource:0}: Error finding container 630f2c2b125e02922b55679b9483bad40bb7affb105b221fffbe651cd1528d15: Status 404 returned error can't find the container with id 630f2c2b125e02922b55679b9483bad40bb7affb105b221fffbe651cd1528d15 Jul 07 11:21:24.247960 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:24.247884 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" event={"ID":"5cf720f3-0e4c-4731-821c-087de85bacf8","Type":"ContainerStarted","Data":"630f2c2b125e02922b55679b9483bad40bb7affb105b221fffbe651cd1528d15"} Jul 07 11:21:24.786478 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:24.786446 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv"] Jul 07 11:21:24.786644 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:24.786611 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" podUID="c1a2da85-c2de-4e2a-9b66-36755b69e8b1" containerName="discovery" containerID="cri-o://741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9" gracePeriod=30 Jul 07 11:21:24.792543 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:24.792515 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-v5gtz\" (UID: \"0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:21:24.794750 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:24.794724 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-v5gtz\" (UID: \"0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:21:24.868917 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:24.868880 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:21:25.112116 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.112090 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:25.195128 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.195100 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-local-certs\") pod \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " Jul 07 11:21:25.195302 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.195189 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-token\") pod \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " Jul 07 11:21:25.195302 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.195231 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-dns-cert\") pod \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " Jul 07 11:21:25.195302 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.195270 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44hmm\" (UniqueName: \"kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-kube-api-access-44hmm\") pod \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " Jul 07 11:21:25.195561 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.195540 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-ca-configmap\") pod \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " Jul 07 11:21:25.195626 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.195613 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-kubeconfig\") pod \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " Jul 07 11:21:25.195685 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.195650 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-cacerts\") pod \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\" (UID: \"c1a2da85-c2de-4e2a-9b66-36755b69e8b1\") " Jul 07 11:21:25.195960 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.195904 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "c1a2da85-c2de-4e2a-9b66-36755b69e8b1" (UID: "c1a2da85-c2de-4e2a-9b66-36755b69e8b1"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:21:25.196898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.196017 2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-ca-configmap\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:21:25.198175 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.198147 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1a2da85-c2de-4e2a-9b66-36755b69e8b1" (UID: "c1a2da85-c2de-4e2a-9b66-36755b69e8b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:21:25.198300 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.198182 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "c1a2da85-c2de-4e2a-9b66-36755b69e8b1" (UID: "c1a2da85-c2de-4e2a-9b66-36755b69e8b1"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:21:25.198511 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.198483 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-cacerts" (OuterVolumeSpecName: "cacerts") pod "c1a2da85-c2de-4e2a-9b66-36755b69e8b1" (UID: "c1a2da85-c2de-4e2a-9b66-36755b69e8b1"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:21:25.198511 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.198502 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-local-certs" (OuterVolumeSpecName: "local-certs") pod "c1a2da85-c2de-4e2a-9b66-36755b69e8b1" (UID: "c1a2da85-c2de-4e2a-9b66-36755b69e8b1"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:21:25.198667 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.198588 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "c1a2da85-c2de-4e2a-9b66-36755b69e8b1" (UID: "c1a2da85-c2de-4e2a-9b66-36755b69e8b1"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:21:25.198957 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.198938 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-kube-api-access-44hmm" (OuterVolumeSpecName: "kube-api-access-44hmm") pod "c1a2da85-c2de-4e2a-9b66-36755b69e8b1" (UID: "c1a2da85-c2de-4e2a-9b66-36755b69e8b1"). InnerVolumeSpecName "kube-api-access-44hmm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:21:25.255052 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.255015 2585 generic.go:356] "Generic (PLEG): container finished" podID="c1a2da85-c2de-4e2a-9b66-36755b69e8b1" containerID="741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9" exitCode=0 Jul 07 11:21:25.255052 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.255039 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" Jul 07 11:21:25.255526 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.255079 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" event={"ID":"c1a2da85-c2de-4e2a-9b66-36755b69e8b1","Type":"ContainerDied","Data":"741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9"} Jul 07 11:21:25.255651 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.255562 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv" event={"ID":"c1a2da85-c2de-4e2a-9b66-36755b69e8b1","Type":"ContainerDied","Data":"245ded488de5d05e9804b7214069f65d588031753ec9c11da8cd3b17422d613c"} Jul 07 11:21:25.255651 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.255614 2585 scope.go:117] "RemoveContainer" containerID="741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9" Jul 07 11:21:25.264277 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:21:25.264174 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0ee8a0df_9dbe_45d9_b34d_a07b259bd5fb.slice/crio-a6eca47071640a22e9f4e3824423bf89840594667697cb2d441d864e38d020fa WatchSource:0}: Error finding container a6eca47071640a22e9f4e3824423bf89840594667697cb2d441d864e38d020fa: Status 404 returned error can't find the container with id a6eca47071640a22e9f4e3824423bf89840594667697cb2d441d864e38d020fa Jul 07 11:21:25.267062 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.267036 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz"] Jul 07 11:21:25.276922 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.276902 2585 scope.go:117] "RemoveContainer" containerID="741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9" Jul 07 11:21:25.277292 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:21:25.277222 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9\": container with ID starting with 741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9 not found: ID does not exist" containerID="741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9" Jul 07 11:21:25.277292 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.277257 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9"} err="failed to get container status \"741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9\": rpc error: code = NotFound desc = could not find container \"741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9\": container with ID starting with 741ad83f285d667a1e99bd9c60c79584924ca696dc5b61f9e399ed048dec6ac9 not found: ID does not exist" Jul 07 11:21:25.297369 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.297320 2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-csr-dns-cert\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:21:25.297511 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.297376 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44hmm\" (UniqueName: \"kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-kube-api-access-44hmm\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:21:25.297511 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.297392 2585 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-kubeconfig\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:21:25.297511 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.297403 2585 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-cacerts\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:21:25.297511 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.297412 2585 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-local-certs\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:21:25.297511 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.297420 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1a2da85-c2de-4e2a-9b66-36755b69e8b1-istio-token\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:21:25.300769 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.300735 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv"] Jul 07 11:21:25.305830 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.305799 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2dwcv"] Jul 07 11:21:25.676801 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:25.676768 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1a2da85-c2de-4e2a-9b66-36755b69e8b1" path="/var/lib/kubelet/pods/c1a2da85-c2de-4e2a-9b66-36755b69e8b1/volumes" Jul 07 11:21:26.261518 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:26.261474 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" event={"ID":"0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb","Type":"ContainerStarted","Data":"a6eca47071640a22e9f4e3824423bf89840594667697cb2d441d864e38d020fa"} Jul 07 11:21:28.276814 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:28.276778 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" event={"ID":"5cf720f3-0e4c-4731-821c-087de85bacf8","Type":"ContainerStarted","Data":"2b1235059cc0820be71e0bf2cb7c692703a39136d27b95c0accdd730f8a90112"} Jul 07 11:21:28.277195 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:28.276918 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" Jul 07 11:21:28.297412 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:28.297361 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" podStartSLOduration=3.193558439 podStartE2EDuration="6.297329797s" totalImagesPullingTime="3.103771358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:28.295549808 +0000 UTC m=+345.883416299" watchObservedRunningTime="2026-07-07 11:21:28.297329797 +0000 UTC m=+345.885196286" Jul 07 11:21:29.227925 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:29.227897 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-ld52x" Jul 07 11:21:29.229812 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:29.229790 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-4vwpx" Jul 07 11:21:29.284287 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:29.284249 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" event={"ID":"0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb","Type":"ContainerStarted","Data":"541f8065588d2f73fe56b8ba311091ebf4f92714ac3a5625992cb93a10e36e95"} Jul 07 11:21:29.284766 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:29.284464 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:21:29.303206 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:29.303156 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" podStartSLOduration=3.257636617 podStartE2EDuration="6.303138684s" totalImagesPullingTime="3.045502067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:29.301514645 +0000 UTC m=+346.889381137" watchObservedRunningTime="2026-07-07 11:21:29.303138684 +0000 UTC m=+346.891005173" Jul 07 11:21:59.289376 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:21:59.289332 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4r68v" Jul 07 11:22:00.292775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:00.292743 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v5gtz" Jul 07 11:22:11.784341 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:11.784307 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m4dwp"] Jul 07 11:22:11.784734 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:11.784644 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1a2da85-c2de-4e2a-9b66-36755b69e8b1" containerName="discovery" Jul 07 11:22:11.784734 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:11.784655 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1a2da85-c2de-4e2a-9b66-36755b69e8b1" containerName="discovery" Jul 07 11:22:11.784734 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:11.784725 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1a2da85-c2de-4e2a-9b66-36755b69e8b1" containerName="discovery" Jul 07 11:22:11.786770 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:11.786752 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" Jul 07 11:22:11.789529 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:11.789504 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-tmgf5\"" Jul 07 11:22:11.789691 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:11.789505 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 11:22:11.789767 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:11.789579 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 11:22:11.799262 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:11.799237 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m4dwp"] Jul 07 11:22:11.901930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:11.901901 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w94vb\" (UniqueName: \"kubernetes.io/projected/b84e357c-91e6-4f69-ab86-d711c80a85f4-kube-api-access-w94vb\") pod \"kuadrant-operator-catalog-m4dwp\" (UID: \"b84e357c-91e6-4f69-ab86-d711c80a85f4\") " pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" Jul 07 11:22:12.002594 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.002560 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w94vb\" (UniqueName: \"kubernetes.io/projected/b84e357c-91e6-4f69-ab86-d711c80a85f4-kube-api-access-w94vb\") pod \"kuadrant-operator-catalog-m4dwp\" (UID: \"b84e357c-91e6-4f69-ab86-d711c80a85f4\") " pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" Jul 07 11:22:12.011854 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.011817 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w94vb\" (UniqueName: \"kubernetes.io/projected/b84e357c-91e6-4f69-ab86-d711c80a85f4-kube-api-access-w94vb\") pod \"kuadrant-operator-catalog-m4dwp\" (UID: \"b84e357c-91e6-4f69-ab86-d711c80a85f4\") " pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" Jul 07 11:22:12.097703 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.097604 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" Jul 07 11:22:12.140894 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.140862 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m4dwp"] Jul 07 11:22:12.231266 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.231241 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m4dwp"] Jul 07 11:22:12.232876 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:22:12.232841 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb84e357c_91e6_4f69_ab86_d711c80a85f4.slice/crio-668d1dc24ea9c6b9a454a2d597955c8dc08ac90bd5c218825184877a8fe2f359 WatchSource:0}: Error finding container 668d1dc24ea9c6b9a454a2d597955c8dc08ac90bd5c218825184877a8fe2f359: Status 404 returned error can't find the container with id 668d1dc24ea9c6b9a454a2d597955c8dc08ac90bd5c218825184877a8fe2f359 Jul 07 11:22:12.351746 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.351667 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pzqg4"] Jul 07 11:22:12.354502 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.354487 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" Jul 07 11:22:12.362696 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.362674 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pzqg4"] Jul 07 11:22:12.406450 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.406415 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d65c\" (UniqueName: \"kubernetes.io/projected/101c4a72-98a9-4b15-86c3-6e8868f92247-kube-api-access-8d65c\") pod \"kuadrant-operator-catalog-pzqg4\" (UID: \"101c4a72-98a9-4b15-86c3-6e8868f92247\") " pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" Jul 07 11:22:12.488397 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.488360 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" event={"ID":"b84e357c-91e6-4f69-ab86-d711c80a85f4","Type":"ContainerStarted","Data":"668d1dc24ea9c6b9a454a2d597955c8dc08ac90bd5c218825184877a8fe2f359"} Jul 07 11:22:12.507613 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.507587 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d65c\" (UniqueName: \"kubernetes.io/projected/101c4a72-98a9-4b15-86c3-6e8868f92247-kube-api-access-8d65c\") pod \"kuadrant-operator-catalog-pzqg4\" (UID: \"101c4a72-98a9-4b15-86c3-6e8868f92247\") " pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" Jul 07 11:22:12.516255 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.516230 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d65c\" (UniqueName: \"kubernetes.io/projected/101c4a72-98a9-4b15-86c3-6e8868f92247-kube-api-access-8d65c\") pod \"kuadrant-operator-catalog-pzqg4\" (UID: \"101c4a72-98a9-4b15-86c3-6e8868f92247\") " pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" Jul 07 11:22:12.664902 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.664866 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" Jul 07 11:22:12.818229 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:12.818206 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pzqg4"] Jul 07 11:22:12.853563 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:22:12.853520 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod101c4a72_98a9_4b15_86c3_6e8868f92247.slice/crio-9f3c6ea31283e5a967c810e9b1a2f5e1f2689c2b99bae6d459f500722ec9eb4f WatchSource:0}: Error finding container 9f3c6ea31283e5a967c810e9b1a2f5e1f2689c2b99bae6d459f500722ec9eb4f: Status 404 returned error can't find the container with id 9f3c6ea31283e5a967c810e9b1a2f5e1f2689c2b99bae6d459f500722ec9eb4f Jul 07 11:22:13.495220 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:13.495182 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" event={"ID":"101c4a72-98a9-4b15-86c3-6e8868f92247","Type":"ContainerStarted","Data":"9f3c6ea31283e5a967c810e9b1a2f5e1f2689c2b99bae6d459f500722ec9eb4f"} Jul 07 11:22:14.501951 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:14.501911 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" event={"ID":"101c4a72-98a9-4b15-86c3-6e8868f92247","Type":"ContainerStarted","Data":"9c33dc2596f85dd564cbf7356453284f65f1f8fb33941f568e5a8b5fe0389f7e"} Jul 07 11:22:14.503491 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:14.503468 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" event={"ID":"b84e357c-91e6-4f69-ab86-d711c80a85f4","Type":"ContainerStarted","Data":"3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f"} Jul 07 11:22:14.503598 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:14.503544 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" podUID="b84e357c-91e6-4f69-ab86-d711c80a85f4" containerName="registry-server" containerID="cri-o://3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f" gracePeriod=2 Jul 07 11:22:14.522221 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:14.522171 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" podStartSLOduration=1.068107797 podStartE2EDuration="2.522151738s" totalImagesPullingTime="1.454043941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:22:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:22:14.520103379 +0000 UTC m=+392.107969871" watchObservedRunningTime="2026-07-07 11:22:14.522151738 +0000 UTC m=+392.110018228" Jul 07 11:22:14.540529 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:14.540478 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" podStartSLOduration=1.468350897 podStartE2EDuration="3.540457801s" totalImagesPullingTime="2.072106904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:22:14.539015958 +0000 UTC m=+392.126882450" watchObservedRunningTime="2026-07-07 11:22:14.540457801 +0000 UTC m=+392.128324292" Jul 07 11:22:14.742440 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:14.742414 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" Jul 07 11:22:14.827815 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:14.827783 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w94vb\" (UniqueName: \"kubernetes.io/projected/b84e357c-91e6-4f69-ab86-d711c80a85f4-kube-api-access-w94vb\") pod \"b84e357c-91e6-4f69-ab86-d711c80a85f4\" (UID: \"b84e357c-91e6-4f69-ab86-d711c80a85f4\") " Jul 07 11:22:14.829961 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:14.829937 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b84e357c-91e6-4f69-ab86-d711c80a85f4-kube-api-access-w94vb" (OuterVolumeSpecName: "kube-api-access-w94vb") pod "b84e357c-91e6-4f69-ab86-d711c80a85f4" (UID: "b84e357c-91e6-4f69-ab86-d711c80a85f4"). InnerVolumeSpecName "kube-api-access-w94vb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:22:14.928815 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:14.928735 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w94vb\" (UniqueName: \"kubernetes.io/projected/b84e357c-91e6-4f69-ab86-d711c80a85f4-kube-api-access-w94vb\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:15.509745 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:15.509714 2585 generic.go:356] "Generic (PLEG): container finished" podID="b84e357c-91e6-4f69-ab86-d711c80a85f4" containerID="3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f" exitCode=0 Jul 07 11:22:15.510121 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:15.509756 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" Jul 07 11:22:15.510121 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:15.509796 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" event={"ID":"b84e357c-91e6-4f69-ab86-d711c80a85f4","Type":"ContainerDied","Data":"3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f"} Jul 07 11:22:15.510121 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:15.509844 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m4dwp" event={"ID":"b84e357c-91e6-4f69-ab86-d711c80a85f4","Type":"ContainerDied","Data":"668d1dc24ea9c6b9a454a2d597955c8dc08ac90bd5c218825184877a8fe2f359"} Jul 07 11:22:15.510121 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:15.509867 2585 scope.go:117] "RemoveContainer" containerID="3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f" Jul 07 11:22:15.519233 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:15.519213 2585 scope.go:117] "RemoveContainer" containerID="3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f" Jul 07 11:22:15.519505 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:22:15.519486 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f\": container with ID starting with 3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f not found: ID does not exist" containerID="3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f" Jul 07 11:22:15.519570 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:15.519514 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f"} err="failed to get container status \"3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f\": rpc error: code = NotFound desc = could not find container \"3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f\": container with ID starting with 3e26f175a4ea3ec6f261f7396bbe221fc868bf2f980d03eadde155ddf50f6a7f not found: ID does not exist" Jul 07 11:22:15.531551 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:15.531528 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m4dwp"] Jul 07 11:22:15.533152 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:15.533132 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m4dwp"] Jul 07 11:22:15.674903 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:15.674873 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b84e357c-91e6-4f69-ab86-d711c80a85f4" path="/var/lib/kubelet/pods/b84e357c-91e6-4f69-ab86-d711c80a85f4/volumes" Jul 07 11:22:22.665960 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:22.665925 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" Jul 07 11:22:22.666405 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:22.665981 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" Jul 07 11:22:22.689154 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:22.689129 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" Jul 07 11:22:23.570548 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:23.570521 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-pzqg4" Jul 07 11:22:27.817127 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:27.817090 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg"] Jul 07 11:22:27.817541 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:27.817449 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b84e357c-91e6-4f69-ab86-d711c80a85f4" containerName="registry-server" Jul 07 11:22:27.817541 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:27.817463 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b84e357c-91e6-4f69-ab86-d711c80a85f4" containerName="registry-server" Jul 07 11:22:27.817541 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:27.817526 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b84e357c-91e6-4f69-ab86-d711c80a85f4" containerName="registry-server" Jul 07 11:22:27.822183 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:27.822163 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:27.825259 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:27.825240 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-w7m48\"" Jul 07 11:22:27.833767 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:27.833749 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg"] Jul 07 11:22:27.943372 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:27.943313 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:27.943547 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:27.943383 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fnjb\" (UniqueName: \"kubernetes.io/projected/e4750739-72ca-4709-bd60-0df6c7551919-kube-api-access-5fnjb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:27.943547 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:27.943433 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:28.044165 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.044128 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:28.044165 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.044169 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fnjb\" (UniqueName: \"kubernetes.io/projected/e4750739-72ca-4709-bd60-0df6c7551919-kube-api-access-5fnjb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:28.044386 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.044298 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:28.044596 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.044578 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:28.044662 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.044647 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:28.053972 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.053945 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fnjb\" (UniqueName: \"kubernetes.io/projected/e4750739-72ca-4709-bd60-0df6c7551919-kube-api-access-5fnjb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:28.131177 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.131147 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:28.273489 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.273464 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg"] Jul 07 11:22:28.275971 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:22:28.275941 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4750739_72ca_4709_bd60_0df6c7551919.slice/crio-58ed54b343144f0d2949114bd2f159994a3e88a8c9c3551f0251fdd9fd244828 WatchSource:0}: Error finding container 58ed54b343144f0d2949114bd2f159994a3e88a8c9c3551f0251fdd9fd244828: Status 404 returned error can't find the container with id 58ed54b343144f0d2949114bd2f159994a3e88a8c9c3551f0251fdd9fd244828 Jul 07 11:22:28.402056 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.402029 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92"] Jul 07 11:22:28.405526 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.405505 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:28.415013 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.414993 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92"] Jul 07 11:22:28.549475 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.549433 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8xgk\" (UniqueName: \"kubernetes.io/projected/a860b85d-fd65-46df-9a27-0284d3091631-kube-api-access-g8xgk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:28.549475 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.549479 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:28.549689 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.549552 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:28.573598 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.573562 2585 generic.go:356] "Generic (PLEG): container finished" podID="e4750739-72ca-4709-bd60-0df6c7551919" containerID="701a1550c868ec7b49dd5ef35e6be428184292affcd85eff157b0ee5317d66c8" exitCode=0 Jul 07 11:22:28.573732 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.573605 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" event={"ID":"e4750739-72ca-4709-bd60-0df6c7551919","Type":"ContainerDied","Data":"701a1550c868ec7b49dd5ef35e6be428184292affcd85eff157b0ee5317d66c8"} Jul 07 11:22:28.573732 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.573651 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" event={"ID":"e4750739-72ca-4709-bd60-0df6c7551919","Type":"ContainerStarted","Data":"58ed54b343144f0d2949114bd2f159994a3e88a8c9c3551f0251fdd9fd244828"} Jul 07 11:22:28.651019 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.650928 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8xgk\" (UniqueName: \"kubernetes.io/projected/a860b85d-fd65-46df-9a27-0284d3091631-kube-api-access-g8xgk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:28.651019 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.650970 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:28.651019 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.650995 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:28.651424 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.651406 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:28.651468 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.651437 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:28.661179 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.661149 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8xgk\" (UniqueName: \"kubernetes.io/projected/a860b85d-fd65-46df-9a27-0284d3091631-kube-api-access-g8xgk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:28.761650 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.761619 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:28.807524 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.807489 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948"] Jul 07 11:22:28.812267 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.812241 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:28.822776 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.822750 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948"] Jul 07 11:22:28.896982 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.896952 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92"] Jul 07 11:22:28.898477 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:22:28.898440 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda860b85d_fd65_46df_9a27_0284d3091631.slice/crio-c8494c2b033fb1978e7a9b09ee9cf2fd053b9052d269bcf5c650b07fa67b0882 WatchSource:0}: Error finding container c8494c2b033fb1978e7a9b09ee9cf2fd053b9052d269bcf5c650b07fa67b0882: Status 404 returned error can't find the container with id c8494c2b033fb1978e7a9b09ee9cf2fd053b9052d269bcf5c650b07fa67b0882 Jul 07 11:22:28.953483 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.953462 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:28.953563 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.953493 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4nbn\" (UniqueName: \"kubernetes.io/projected/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-kube-api-access-r4nbn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:28.953602 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:28.953578 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:29.054148 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.054117 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:29.054148 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.054154 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4nbn\" (UniqueName: \"kubernetes.io/projected/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-kube-api-access-r4nbn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:29.054371 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.054196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:29.054582 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.054562 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:29.054658 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.054569 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:29.063741 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.063718 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4nbn\" (UniqueName: \"kubernetes.io/projected/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-kube-api-access-r4nbn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:29.128628 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.128597 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:29.192660 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.192632 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r"] Jul 07 11:22:29.196097 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.196077 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:29.205332 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.205304 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r"] Jul 07 11:22:29.273826 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.273804 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948"] Jul 07 11:22:29.333102 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:22:29.333070 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d3b15d_9fbc_4bb6_80c7_fda597f25c8a.slice/crio-8a6461f69d6eda839ae7e7a59b12db4e45d32d687362e2670ccbbea74b7f012f WatchSource:0}: Error finding container 8a6461f69d6eda839ae7e7a59b12db4e45d32d687362e2670ccbbea74b7f012f: Status 404 returned error can't find the container with id 8a6461f69d6eda839ae7e7a59b12db4e45d32d687362e2670ccbbea74b7f012f Jul 07 11:22:29.357155 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.357126 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:29.357298 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.357244 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:29.357298 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.357275 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lslfn\" (UniqueName: \"kubernetes.io/projected/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-kube-api-access-lslfn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:29.458056 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.458020 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:29.458209 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.458074 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lslfn\" (UniqueName: \"kubernetes.io/projected/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-kube-api-access-lslfn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:29.458273 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.458233 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:29.458438 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.458413 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:29.458542 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.458527 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:29.468424 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.468394 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lslfn\" (UniqueName: \"kubernetes.io/projected/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-kube-api-access-lslfn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:29.506877 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.506847 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:29.580771 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.580740 2585 generic.go:356] "Generic (PLEG): container finished" podID="33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" containerID="662ddfd7520d49623dc133c7124a6935d6747f02d2e07523dfa1124dec81cd20" exitCode=0 Jul 07 11:22:29.580944 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.580807 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" event={"ID":"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a","Type":"ContainerDied","Data":"662ddfd7520d49623dc133c7124a6935d6747f02d2e07523dfa1124dec81cd20"} Jul 07 11:22:29.580944 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.580866 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" event={"ID":"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a","Type":"ContainerStarted","Data":"8a6461f69d6eda839ae7e7a59b12db4e45d32d687362e2670ccbbea74b7f012f"} Jul 07 11:22:29.582610 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.582583 2585 generic.go:356] "Generic (PLEG): container finished" podID="a860b85d-fd65-46df-9a27-0284d3091631" containerID="b33587c3c3dc4f21bbbf55cd033c03525578b55a4eb985eef3ca6bbc25187c3b" exitCode=0 Jul 07 11:22:29.582712 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.582661 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" event={"ID":"a860b85d-fd65-46df-9a27-0284d3091631","Type":"ContainerDied","Data":"b33587c3c3dc4f21bbbf55cd033c03525578b55a4eb985eef3ca6bbc25187c3b"} Jul 07 11:22:29.582782 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.582713 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" event={"ID":"a860b85d-fd65-46df-9a27-0284d3091631","Type":"ContainerStarted","Data":"c8494c2b033fb1978e7a9b09ee9cf2fd053b9052d269bcf5c650b07fa67b0882"} Jul 07 11:22:29.584837 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.584813 2585 generic.go:356] "Generic (PLEG): container finished" podID="e4750739-72ca-4709-bd60-0df6c7551919" containerID="471a308758aebda175528c75e256e13e547e89e0e5784c6ac74a7d725167a549" exitCode=0 Jul 07 11:22:29.584940 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.584857 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" event={"ID":"e4750739-72ca-4709-bd60-0df6c7551919","Type":"ContainerDied","Data":"471a308758aebda175528c75e256e13e547e89e0e5784c6ac74a7d725167a549"} Jul 07 11:22:29.648382 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:29.648360 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r"] Jul 07 11:22:29.650198 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:22:29.650168 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f054cf_9787_4d84_bcd0_c6e32623e0eb.slice/crio-f253f475a100539584704b97a8f11d33a1eb549abd1fe25de34178aea738923a WatchSource:0}: Error finding container f253f475a100539584704b97a8f11d33a1eb549abd1fe25de34178aea738923a: Status 404 returned error can't find the container with id f253f475a100539584704b97a8f11d33a1eb549abd1fe25de34178aea738923a Jul 07 11:22:30.591541 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:30.591459 2585 generic.go:356] "Generic (PLEG): container finished" podID="e4750739-72ca-4709-bd60-0df6c7551919" containerID="eca5b8e5d89721878681e8ede6da5f97dd5d1553f2f9700efe4cc4d31a610a5d" exitCode=0 Jul 07 11:22:30.591915 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:30.591539 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" event={"ID":"e4750739-72ca-4709-bd60-0df6c7551919","Type":"ContainerDied","Data":"eca5b8e5d89721878681e8ede6da5f97dd5d1553f2f9700efe4cc4d31a610a5d"} Jul 07 11:22:30.593247 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:30.593225 2585 generic.go:356] "Generic (PLEG): container finished" podID="33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" containerID="e553285c119dd8b4f344d84a438352f4108b34049e8ead28c3db93e9204969d5" exitCode=0 Jul 07 11:22:30.593396 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:30.593253 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" event={"ID":"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a","Type":"ContainerDied","Data":"e553285c119dd8b4f344d84a438352f4108b34049e8ead28c3db93e9204969d5"} Jul 07 11:22:30.595011 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:30.594989 2585 generic.go:356] "Generic (PLEG): container finished" podID="a860b85d-fd65-46df-9a27-0284d3091631" containerID="023653e80899a4e0ee6cbb02f3a130b0c5bd4eaf260ca1339b9f271195de8e25" exitCode=0 Jul 07 11:22:30.595123 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:30.595073 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" event={"ID":"a860b85d-fd65-46df-9a27-0284d3091631","Type":"ContainerDied","Data":"023653e80899a4e0ee6cbb02f3a130b0c5bd4eaf260ca1339b9f271195de8e25"} Jul 07 11:22:30.596638 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:30.596617 2585 generic.go:356] "Generic (PLEG): container finished" podID="c7f054cf-9787-4d84-bcd0-c6e32623e0eb" containerID="7866ffac4aea7d5a874d5ec8e312e8361dc4e65da7544e9c68bfd2def8f698ab" exitCode=0 Jul 07 11:22:30.596751 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:30.596680 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" event={"ID":"c7f054cf-9787-4d84-bcd0-c6e32623e0eb","Type":"ContainerDied","Data":"7866ffac4aea7d5a874d5ec8e312e8361dc4e65da7544e9c68bfd2def8f698ab"} Jul 07 11:22:30.596751 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:30.596715 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" event={"ID":"c7f054cf-9787-4d84-bcd0-c6e32623e0eb","Type":"ContainerStarted","Data":"f253f475a100539584704b97a8f11d33a1eb549abd1fe25de34178aea738923a"} Jul 07 11:22:31.603891 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.603803 2585 generic.go:356] "Generic (PLEG): container finished" podID="33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" containerID="c5dfb0690ed4b3fdaef7b002eed783885b4b00de9f50b5d3c56ee2f65431aba1" exitCode=0 Jul 07 11:22:31.604316 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.603892 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" event={"ID":"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a","Type":"ContainerDied","Data":"c5dfb0690ed4b3fdaef7b002eed783885b4b00de9f50b5d3c56ee2f65431aba1"} Jul 07 11:22:31.605811 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.605792 2585 generic.go:356] "Generic (PLEG): container finished" podID="a860b85d-fd65-46df-9a27-0284d3091631" containerID="28954d6a831e90f874a6bca7fb97c6d9e9c7f425ef0bc297eba3b660a95aa429" exitCode=0 Jul 07 11:22:31.605910 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.605876 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" event={"ID":"a860b85d-fd65-46df-9a27-0284d3091631","Type":"ContainerDied","Data":"28954d6a831e90f874a6bca7fb97c6d9e9c7f425ef0bc297eba3b660a95aa429"} Jul 07 11:22:31.607483 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.607462 2585 generic.go:356] "Generic (PLEG): container finished" podID="c7f054cf-9787-4d84-bcd0-c6e32623e0eb" containerID="1b111aad5848b46fdcadd1867f6a8965084cafb61d8c1f4608a528dc34a04634" exitCode=0 Jul 07 11:22:31.607580 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.607524 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" event={"ID":"c7f054cf-9787-4d84-bcd0-c6e32623e0eb","Type":"ContainerDied","Data":"1b111aad5848b46fdcadd1867f6a8965084cafb61d8c1f4608a528dc34a04634"} Jul 07 11:22:31.749651 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.749627 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:31.884749 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.884728 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-bundle\") pod \"e4750739-72ca-4709-bd60-0df6c7551919\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " Jul 07 11:22:31.884934 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.884774 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fnjb\" (UniqueName: \"kubernetes.io/projected/e4750739-72ca-4709-bd60-0df6c7551919-kube-api-access-5fnjb\") pod \"e4750739-72ca-4709-bd60-0df6c7551919\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " Jul 07 11:22:31.884934 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.884825 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-util\") pod \"e4750739-72ca-4709-bd60-0df6c7551919\" (UID: \"e4750739-72ca-4709-bd60-0df6c7551919\") " Jul 07 11:22:31.885252 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.885228 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-bundle" (OuterVolumeSpecName: "bundle") pod "e4750739-72ca-4709-bd60-0df6c7551919" (UID: "e4750739-72ca-4709-bd60-0df6c7551919"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:22:31.886731 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.886699 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4750739-72ca-4709-bd60-0df6c7551919-kube-api-access-5fnjb" (OuterVolumeSpecName: "kube-api-access-5fnjb") pod "e4750739-72ca-4709-bd60-0df6c7551919" (UID: "e4750739-72ca-4709-bd60-0df6c7551919"). InnerVolumeSpecName "kube-api-access-5fnjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:22:31.890805 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.890779 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-util" (OuterVolumeSpecName: "util") pod "e4750739-72ca-4709-bd60-0df6c7551919" (UID: "e4750739-72ca-4709-bd60-0df6c7551919"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:22:31.985897 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.985866 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-bundle\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:31.985897 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.985895 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fnjb\" (UniqueName: \"kubernetes.io/projected/e4750739-72ca-4709-bd60-0df6c7551919-kube-api-access-5fnjb\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:31.985897 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:31.985905 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4750739-72ca-4709-bd60-0df6c7551919-util\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:32.614858 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.614822 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" Jul 07 11:22:32.614858 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.614841 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg" event={"ID":"e4750739-72ca-4709-bd60-0df6c7551919","Type":"ContainerDied","Data":"58ed54b343144f0d2949114bd2f159994a3e88a8c9c3551f0251fdd9fd244828"} Jul 07 11:22:32.615382 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.614888 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58ed54b343144f0d2949114bd2f159994a3e88a8c9c3551f0251fdd9fd244828" Jul 07 11:22:32.617048 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.617022 2585 generic.go:356] "Generic (PLEG): container finished" podID="c7f054cf-9787-4d84-bcd0-c6e32623e0eb" containerID="8a09223e1a0231f45dafc7ca365c9d45fea06ffa4a67e025fcd1e86a4cb59834" exitCode=0 Jul 07 11:22:32.617155 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.617103 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" event={"ID":"c7f054cf-9787-4d84-bcd0-c6e32623e0eb","Type":"ContainerDied","Data":"8a09223e1a0231f45dafc7ca365c9d45fea06ffa4a67e025fcd1e86a4cb59834"} Jul 07 11:22:32.796795 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.796767 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:32.799007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.798991 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:32.894437 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.894409 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-bundle\") pod \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " Jul 07 11:22:32.894591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.894464 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8xgk\" (UniqueName: \"kubernetes.io/projected/a860b85d-fd65-46df-9a27-0284d3091631-kube-api-access-g8xgk\") pod \"a860b85d-fd65-46df-9a27-0284d3091631\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " Jul 07 11:22:32.894591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.894489 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-bundle\") pod \"a860b85d-fd65-46df-9a27-0284d3091631\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " Jul 07 11:22:32.894591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.894505 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-util\") pod \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " Jul 07 11:22:32.894591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.894533 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-util\") pod \"a860b85d-fd65-46df-9a27-0284d3091631\" (UID: \"a860b85d-fd65-46df-9a27-0284d3091631\") " Jul 07 11:22:32.894591 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.894548 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4nbn\" (UniqueName: \"kubernetes.io/projected/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-kube-api-access-r4nbn\") pod \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\" (UID: \"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a\") " Jul 07 11:22:32.895043 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.895012 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-bundle" (OuterVolumeSpecName: "bundle") pod "33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" (UID: "33d3b15d-9fbc-4bb6-80c7-fda597f25c8a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:22:32.895303 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.895277 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-bundle" (OuterVolumeSpecName: "bundle") pod "a860b85d-fd65-46df-9a27-0284d3091631" (UID: "a860b85d-fd65-46df-9a27-0284d3091631"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:22:32.896904 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.896875 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-kube-api-access-r4nbn" (OuterVolumeSpecName: "kube-api-access-r4nbn") pod "33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" (UID: "33d3b15d-9fbc-4bb6-80c7-fda597f25c8a"). InnerVolumeSpecName "kube-api-access-r4nbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:22:32.897224 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.897197 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a860b85d-fd65-46df-9a27-0284d3091631-kube-api-access-g8xgk" (OuterVolumeSpecName: "kube-api-access-g8xgk") pod "a860b85d-fd65-46df-9a27-0284d3091631" (UID: "a860b85d-fd65-46df-9a27-0284d3091631"). InnerVolumeSpecName "kube-api-access-g8xgk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:22:32.901102 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.901070 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-util" (OuterVolumeSpecName: "util") pod "33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" (UID: "33d3b15d-9fbc-4bb6-80c7-fda597f25c8a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:22:32.901597 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.901560 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-util" (OuterVolumeSpecName: "util") pod "a860b85d-fd65-46df-9a27-0284d3091631" (UID: "a860b85d-fd65-46df-9a27-0284d3091631"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:22:32.995782 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.995752 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-bundle\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:32.995782 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.995777 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8xgk\" (UniqueName: \"kubernetes.io/projected/a860b85d-fd65-46df-9a27-0284d3091631-kube-api-access-g8xgk\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:32.995782 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.995787 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-bundle\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:32.996002 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.995794 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-util\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:32.996002 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.995802 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a860b85d-fd65-46df-9a27-0284d3091631-util\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:32.996002 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:32.995810 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4nbn\" (UniqueName: \"kubernetes.io/projected/33d3b15d-9fbc-4bb6-80c7-fda597f25c8a-kube-api-access-r4nbn\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:33.624448 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.624421 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" Jul 07 11:22:33.624865 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.624419 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948" event={"ID":"33d3b15d-9fbc-4bb6-80c7-fda597f25c8a","Type":"ContainerDied","Data":"8a6461f69d6eda839ae7e7a59b12db4e45d32d687362e2670ccbbea74b7f012f"} Jul 07 11:22:33.624865 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.624554 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a6461f69d6eda839ae7e7a59b12db4e45d32d687362e2670ccbbea74b7f012f" Jul 07 11:22:33.626465 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.626444 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" Jul 07 11:22:33.626600 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.626469 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92" event={"ID":"a860b85d-fd65-46df-9a27-0284d3091631","Type":"ContainerDied","Data":"c8494c2b033fb1978e7a9b09ee9cf2fd053b9052d269bcf5c650b07fa67b0882"} Jul 07 11:22:33.626600 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.626504 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8494c2b033fb1978e7a9b09ee9cf2fd053b9052d269bcf5c650b07fa67b0882" Jul 07 11:22:33.765513 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.765490 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:33.904770 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.904687 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-bundle\") pod \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " Jul 07 11:22:33.904770 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.904753 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lslfn\" (UniqueName: \"kubernetes.io/projected/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-kube-api-access-lslfn\") pod \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " Jul 07 11:22:33.904997 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.904784 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-util\") pod \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\" (UID: \"c7f054cf-9787-4d84-bcd0-c6e32623e0eb\") " Jul 07 11:22:33.905279 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.905243 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-bundle" (OuterVolumeSpecName: "bundle") pod "c7f054cf-9787-4d84-bcd0-c6e32623e0eb" (UID: "c7f054cf-9787-4d84-bcd0-c6e32623e0eb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:22:33.906875 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.906843 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-kube-api-access-lslfn" (OuterVolumeSpecName: "kube-api-access-lslfn") pod "c7f054cf-9787-4d84-bcd0-c6e32623e0eb" (UID: "c7f054cf-9787-4d84-bcd0-c6e32623e0eb"). InnerVolumeSpecName "kube-api-access-lslfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:22:33.909826 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:33.909802 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-util" (OuterVolumeSpecName: "util") pod "c7f054cf-9787-4d84-bcd0-c6e32623e0eb" (UID: "c7f054cf-9787-4d84-bcd0-c6e32623e0eb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:22:34.006161 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:34.006121 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-bundle\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:34.006161 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:34.006153 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lslfn\" (UniqueName: \"kubernetes.io/projected/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-kube-api-access-lslfn\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:34.006161 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:34.006162 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f054cf-9787-4d84-bcd0-c6e32623e0eb-util\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:22:34.633844 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:34.633811 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" event={"ID":"c7f054cf-9787-4d84-bcd0-c6e32623e0eb","Type":"ContainerDied","Data":"f253f475a100539584704b97a8f11d33a1eb549abd1fe25de34178aea738923a"} Jul 07 11:22:34.634220 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:34.633885 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f253f475a100539584704b97a8f11d33a1eb549abd1fe25de34178aea738923a" Jul 07 11:22:34.634220 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:34.633827 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r" Jul 07 11:22:43.997328 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997298 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx"] Jul 07 11:22:43.997708 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997692 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4750739-72ca-4709-bd60-0df6c7551919" containerName="pull" Jul 07 11:22:43.997708 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997704 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4750739-72ca-4709-bd60-0df6c7551919" containerName="pull" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997712 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f054cf-9787-4d84-bcd0-c6e32623e0eb" containerName="extract" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997718 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f054cf-9787-4d84-bcd0-c6e32623e0eb" containerName="extract" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997728 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4750739-72ca-4709-bd60-0df6c7551919" containerName="extract" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997733 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4750739-72ca-4709-bd60-0df6c7551919" containerName="extract" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997741 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" containerName="pull" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997746 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" containerName="pull" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997758 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f054cf-9787-4d84-bcd0-c6e32623e0eb" containerName="pull" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997763 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f054cf-9787-4d84-bcd0-c6e32623e0eb" containerName="pull" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997770 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a860b85d-fd65-46df-9a27-0284d3091631" containerName="util" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997775 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a860b85d-fd65-46df-9a27-0284d3091631" containerName="util" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997780 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" containerName="extract" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997784 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" containerName="extract" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997796 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4750739-72ca-4709-bd60-0df6c7551919" containerName="util" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997801 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4750739-72ca-4709-bd60-0df6c7551919" containerName="util" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997809 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a860b85d-fd65-46df-9a27-0284d3091631" containerName="extract" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997822 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a860b85d-fd65-46df-9a27-0284d3091631" containerName="extract" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997829 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" containerName="util" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997835 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" containerName="util" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997840 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f054cf-9787-4d84-bcd0-c6e32623e0eb" containerName="util" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997845 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f054cf-9787-4d84-bcd0-c6e32623e0eb" containerName="util" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997853 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a860b85d-fd65-46df-9a27-0284d3091631" containerName="pull" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997858 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a860b85d-fd65-46df-9a27-0284d3091631" containerName="pull" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997905 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="33d3b15d-9fbc-4bb6-80c7-fda597f25c8a" containerName="extract" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997912 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4750739-72ca-4709-bd60-0df6c7551919" containerName="extract" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997919 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7f054cf-9787-4d84-bcd0-c6e32623e0eb" containerName="extract" Jul 07 11:22:43.997930 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:43.997925 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a860b85d-fd65-46df-9a27-0284d3091631" containerName="extract" Jul 07 11:22:44.001754 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:44.001738 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:22:44.006848 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:44.006828 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nj875\"" Jul 07 11:22:44.013058 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:44.013037 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx"] Jul 07 11:22:44.098568 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:44.098538 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9kp8\" (UniqueName: \"kubernetes.io/projected/34cf95af-4e4e-48e5-84f2-d5bc47be230b-kube-api-access-q9kp8\") pod \"limitador-operator-controller-manager-859d7b78bd-2kkxx\" (UID: \"34cf95af-4e4e-48e5-84f2-d5bc47be230b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:22:44.199835 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:44.199745 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9kp8\" (UniqueName: \"kubernetes.io/projected/34cf95af-4e4e-48e5-84f2-d5bc47be230b-kube-api-access-q9kp8\") pod \"limitador-operator-controller-manager-859d7b78bd-2kkxx\" (UID: \"34cf95af-4e4e-48e5-84f2-d5bc47be230b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:22:44.245482 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:44.245451 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9kp8\" (UniqueName: \"kubernetes.io/projected/34cf95af-4e4e-48e5-84f2-d5bc47be230b-kube-api-access-q9kp8\") pod \"limitador-operator-controller-manager-859d7b78bd-2kkxx\" (UID: \"34cf95af-4e4e-48e5-84f2-d5bc47be230b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:22:44.311634 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:44.311601 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:22:44.491939 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:44.491916 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx"] Jul 07 11:22:44.494036 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:22:44.494009 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34cf95af_4e4e_48e5_84f2_d5bc47be230b.slice/crio-a0e28344cebae53eeea16105cc944ff70ae7be9069fec6ee935d98f0608ebe9a WatchSource:0}: Error finding container a0e28344cebae53eeea16105cc944ff70ae7be9069fec6ee935d98f0608ebe9a: Status 404 returned error can't find the container with id a0e28344cebae53eeea16105cc944ff70ae7be9069fec6ee935d98f0608ebe9a Jul 07 11:22:44.683025 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:44.682989 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" event={"ID":"34cf95af-4e4e-48e5-84f2-d5bc47be230b","Type":"ContainerStarted","Data":"a0e28344cebae53eeea16105cc944ff70ae7be9069fec6ee935d98f0608ebe9a"} Jul 07 11:22:46.694021 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:46.693981 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" event={"ID":"34cf95af-4e4e-48e5-84f2-d5bc47be230b","Type":"ContainerStarted","Data":"463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f"} Jul 07 11:22:46.694500 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:46.694115 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:22:46.721054 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:46.720998 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" podStartSLOduration=1.873574788 podStartE2EDuration="3.720983178s" totalImagesPullingTime="1.84740839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:22:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:22:46.718209903 +0000 UTC m=+424.306076394" watchObservedRunningTime="2026-07-07 11:22:46.720983178 +0000 UTC m=+424.308849668" Jul 07 11:22:49.142848 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.142810 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns"] Jul 07 11:22:49.148239 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.148217 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:22:49.150842 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.150817 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d8l4g\"" Jul 07 11:22:49.165087 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.165059 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns"] Jul 07 11:22:49.246586 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.246554 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gzjns\" (UID: \"358c8476-1c0d-4bb9-9478-b1e1a43a2a27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:22:49.246586 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.246588 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlfqt\" (UniqueName: \"kubernetes.io/projected/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-kube-api-access-vlfqt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gzjns\" (UID: \"358c8476-1c0d-4bb9-9478-b1e1a43a2a27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:22:49.347069 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.347032 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gzjns\" (UID: \"358c8476-1c0d-4bb9-9478-b1e1a43a2a27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:22:49.347069 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.347069 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlfqt\" (UniqueName: \"kubernetes.io/projected/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-kube-api-access-vlfqt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gzjns\" (UID: \"358c8476-1c0d-4bb9-9478-b1e1a43a2a27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:22:49.347489 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.347471 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gzjns\" (UID: \"358c8476-1c0d-4bb9-9478-b1e1a43a2a27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:22:49.362312 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.362282 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlfqt\" (UniqueName: \"kubernetes.io/projected/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-kube-api-access-vlfqt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gzjns\" (UID: \"358c8476-1c0d-4bb9-9478-b1e1a43a2a27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:22:49.458381 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.458279 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:22:49.597593 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.597565 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns"] Jul 07 11:22:49.599600 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:22:49.599574 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358c8476_1c0d_4bb9_9478_b1e1a43a2a27.slice/crio-57511a6fcd13ac6b97f1ac9e8a8e7a99855d6a48da7920c668b7460d98a7fe5b WatchSource:0}: Error finding container 57511a6fcd13ac6b97f1ac9e8a8e7a99855d6a48da7920c668b7460d98a7fe5b: Status 404 returned error can't find the container with id 57511a6fcd13ac6b97f1ac9e8a8e7a99855d6a48da7920c668b7460d98a7fe5b Jul 07 11:22:49.709825 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:49.709752 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" event={"ID":"358c8476-1c0d-4bb9-9478-b1e1a43a2a27","Type":"ContainerStarted","Data":"57511a6fcd13ac6b97f1ac9e8a8e7a99855d6a48da7920c668b7460d98a7fe5b"} Jul 07 11:22:51.512007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:51.511972 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g"] Jul 07 11:22:51.515385 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:51.515364 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g" Jul 07 11:22:51.518265 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:51.518244 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-455z9\"" Jul 07 11:22:51.525803 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:51.525783 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g"] Jul 07 11:22:51.566564 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:51.566530 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxdbn\" (UniqueName: \"kubernetes.io/projected/13b528bb-09b5-4927-8bc4-a4e592f25e91-kube-api-access-cxdbn\") pod \"authorino-operator-75cc8d5cb7-lsb6g\" (UID: \"13b528bb-09b5-4927-8bc4-a4e592f25e91\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g" Jul 07 11:22:51.667584 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:51.667549 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxdbn\" (UniqueName: \"kubernetes.io/projected/13b528bb-09b5-4927-8bc4-a4e592f25e91-kube-api-access-cxdbn\") pod \"authorino-operator-75cc8d5cb7-lsb6g\" (UID: \"13b528bb-09b5-4927-8bc4-a4e592f25e91\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g" Jul 07 11:22:51.676582 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:51.676555 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxdbn\" (UniqueName: \"kubernetes.io/projected/13b528bb-09b5-4927-8bc4-a4e592f25e91-kube-api-access-cxdbn\") pod \"authorino-operator-75cc8d5cb7-lsb6g\" (UID: \"13b528bb-09b5-4927-8bc4-a4e592f25e91\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g" Jul 07 11:22:51.826712 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:51.826628 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g" Jul 07 11:22:51.975281 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:51.975254 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g"] Jul 07 11:22:51.976987 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:22:51.976958 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b528bb_09b5_4927_8bc4_a4e592f25e91.slice/crio-8863af8b1aef430d661c7a879634bca39d482c30464f4fce09eea3605947cbf4 WatchSource:0}: Error finding container 8863af8b1aef430d661c7a879634bca39d482c30464f4fce09eea3605947cbf4: Status 404 returned error can't find the container with id 8863af8b1aef430d661c7a879634bca39d482c30464f4fce09eea3605947cbf4 Jul 07 11:22:52.727979 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:52.727941 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g" event={"ID":"13b528bb-09b5-4927-8bc4-a4e592f25e91","Type":"ContainerStarted","Data":"8863af8b1aef430d661c7a879634bca39d482c30464f4fce09eea3605947cbf4"} Jul 07 11:22:55.748526 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:55.748486 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g" event={"ID":"13b528bb-09b5-4927-8bc4-a4e592f25e91","Type":"ContainerStarted","Data":"d5cf9c68ba78a6da5b38985814300b5925979c976528b8d0e50e195a7c044700"} Jul 07 11:22:55.749012 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:55.748684 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g" Jul 07 11:22:55.750326 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:55.750296 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" event={"ID":"358c8476-1c0d-4bb9-9478-b1e1a43a2a27","Type":"ContainerStarted","Data":"623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703"} Jul 07 11:22:55.750469 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:55.750389 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:22:55.772188 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:55.772130 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g" podStartSLOduration=1.208855371 podStartE2EDuration="4.772112246s" totalImagesPullingTime="3.563256875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:22:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:22:55.771081912 +0000 UTC m=+433.358948415" watchObservedRunningTime="2026-07-07 11:22:55.772112246 +0000 UTC m=+433.359978736" Jul 07 11:22:55.798713 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:55.798613 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" podStartSLOduration=0.796772454 podStartE2EDuration="6.798597614s" totalImagesPullingTime="6.00182516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:22:55.79652825 +0000 UTC m=+433.384394740" watchObservedRunningTime="2026-07-07 11:22:55.798597614 +0000 UTC m=+433.386464104" Jul 07 11:22:57.700255 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:22:57.700220 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:23:06.757723 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:06.757641 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lsb6g" Jul 07 11:23:06.758177 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:06.757815 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:23:08.780933 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.780900 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns"] Jul 07 11:23:08.786694 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.781447 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" podUID="358c8476-1c0d-4bb9-9478-b1e1a43a2a27" containerName="manager" containerID="cri-o://623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703" gracePeriod=2 Jul 07 11:23:08.805560 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.805531 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns"] Jul 07 11:23:08.823244 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:23:08.823202 2585 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"358c8476-1c0d-4bb9-9478-b1e1a43a2a27\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-gzjns\": pods \"kuadrant-operator-controller-manager-8684c9ff49-gzjns\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:23:08.827091 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.827058 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq"] Jul 07 11:23:08.827511 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.827495 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="358c8476-1c0d-4bb9-9478-b1e1a43a2a27" containerName="manager" Jul 07 11:23:08.827511 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.827512 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="358c8476-1c0d-4bb9-9478-b1e1a43a2a27" containerName="manager" Jul 07 11:23:08.827630 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.827579 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="358c8476-1c0d-4bb9-9478-b1e1a43a2a27" containerName="manager" Jul 07 11:23:08.830684 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.830661 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:08.832007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.831986 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx"] Jul 07 11:23:08.832158 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.832138 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" podUID="34cf95af-4e4e-48e5-84f2-d5bc47be230b" containerName="manager" containerID="cri-o://463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f" gracePeriod=2 Jul 07 11:23:08.834302 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:23:08.834271 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gzjns\" is forbidden: User \"system:node:ip-10-0-135-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-166.ec2.internal' and this object" podUID="358c8476-1c0d-4bb9-9478-b1e1a43a2a27" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:23:08.856135 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.856103 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx"] Jul 07 11:23:08.861017 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.860984 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq"] Jul 07 11:23:08.885207 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.885170 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb"] Jul 07 11:23:08.885701 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.885682 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34cf95af-4e4e-48e5-84f2-d5bc47be230b" containerName="manager" Jul 07 11:23:08.885756 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.885706 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="34cf95af-4e4e-48e5-84f2-d5bc47be230b" containerName="manager" Jul 07 11:23:08.885828 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.885817 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="34cf95af-4e4e-48e5-84f2-d5bc47be230b" containerName="manager" Jul 07 11:23:08.888860 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.888837 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb" Jul 07 11:23:08.903369 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:23:08.903308 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gzjns\" is forbidden: User \"system:node:ip-10-0-135-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-166.ec2.internal' and this object" podUID="358c8476-1c0d-4bb9-9478-b1e1a43a2a27" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:23:08.916006 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.915981 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb"] Jul 07 11:23:08.925396 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.925374 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55d8472a-c726-416d-b095-8658db856f36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69xpq\" (UID: \"55d8472a-c726-416d-b095-8658db856f36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:08.925517 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:08.925425 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdfws\" (UniqueName: \"kubernetes.io/projected/55d8472a-c726-416d-b095-8658db856f36-kube-api-access-cdfws\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69xpq\" (UID: \"55d8472a-c726-416d-b095-8658db856f36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:08.952897 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:23:08.952853 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2kkxx\" is forbidden: User \"system:node:ip-10-0-135-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-166.ec2.internal' and this object" podUID="34cf95af-4e4e-48e5-84f2-d5bc47be230b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:23:09.026890 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.026859 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55d8472a-c726-416d-b095-8658db856f36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69xpq\" (UID: \"55d8472a-c726-416d-b095-8658db856f36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:09.027035 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.026915 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdfws\" (UniqueName: \"kubernetes.io/projected/55d8472a-c726-416d-b095-8658db856f36-kube-api-access-cdfws\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69xpq\" (UID: \"55d8472a-c726-416d-b095-8658db856f36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:09.027035 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.026939 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5725p\" (UniqueName: \"kubernetes.io/projected/c18cde74-7cd5-4bac-9fd5-95ad639e84e7-kube-api-access-5725p\") pod \"limitador-operator-controller-manager-859d7b78bd-jjclb\" (UID: \"c18cde74-7cd5-4bac-9fd5-95ad639e84e7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb" Jul 07 11:23:09.027385 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.027253 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55d8472a-c726-416d-b095-8658db856f36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69xpq\" (UID: \"55d8472a-c726-416d-b095-8658db856f36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:09.041833 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.041765 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdfws\" (UniqueName: \"kubernetes.io/projected/55d8472a-c726-416d-b095-8658db856f36-kube-api-access-cdfws\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69xpq\" (UID: \"55d8472a-c726-416d-b095-8658db856f36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:09.080168 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.080143 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:23:09.082807 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.082786 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:23:09.083549 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:23:09.083527 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gzjns\" is forbidden: User \"system:node:ip-10-0-135-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-166.ec2.internal' and this object" podUID="358c8476-1c0d-4bb9-9478-b1e1a43a2a27" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:23:09.085903 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:23:09.085880 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2kkxx\" is forbidden: User \"system:node:ip-10-0-135-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-166.ec2.internal' and this object" podUID="34cf95af-4e4e-48e5-84f2-d5bc47be230b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:23:09.088531 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:23:09.088504 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gzjns\" is forbidden: User \"system:node:ip-10-0-135-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-166.ec2.internal' and this object" podUID="358c8476-1c0d-4bb9-9478-b1e1a43a2a27" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:23:09.090788 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:23:09.090761 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2kkxx\" is forbidden: User \"system:node:ip-10-0-135-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-166.ec2.internal' and this object" podUID="34cf95af-4e4e-48e5-84f2-d5bc47be230b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:23:09.127407 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.127376 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5725p\" (UniqueName: \"kubernetes.io/projected/c18cde74-7cd5-4bac-9fd5-95ad639e84e7-kube-api-access-5725p\") pod \"limitador-operator-controller-manager-859d7b78bd-jjclb\" (UID: \"c18cde74-7cd5-4bac-9fd5-95ad639e84e7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb" Jul 07 11:23:09.149644 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.149618 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5725p\" (UniqueName: \"kubernetes.io/projected/c18cde74-7cd5-4bac-9fd5-95ad639e84e7-kube-api-access-5725p\") pod \"limitador-operator-controller-manager-859d7b78bd-jjclb\" (UID: \"c18cde74-7cd5-4bac-9fd5-95ad639e84e7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb" Jul 07 11:23:09.205199 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.205160 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:09.212180 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.212163 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb" Jul 07 11:23:09.228316 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.228292 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9kp8\" (UniqueName: \"kubernetes.io/projected/34cf95af-4e4e-48e5-84f2-d5bc47be230b-kube-api-access-q9kp8\") pod \"34cf95af-4e4e-48e5-84f2-d5bc47be230b\" (UID: \"34cf95af-4e4e-48e5-84f2-d5bc47be230b\") " Jul 07 11:23:09.228479 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.228386 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-extensions-socket-volume\") pod \"358c8476-1c0d-4bb9-9478-b1e1a43a2a27\" (UID: \"358c8476-1c0d-4bb9-9478-b1e1a43a2a27\") " Jul 07 11:23:09.228479 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.228452 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlfqt\" (UniqueName: \"kubernetes.io/projected/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-kube-api-access-vlfqt\") pod \"358c8476-1c0d-4bb9-9478-b1e1a43a2a27\" (UID: \"358c8476-1c0d-4bb9-9478-b1e1a43a2a27\") " Jul 07 11:23:09.229066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.229023 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "358c8476-1c0d-4bb9-9478-b1e1a43a2a27" (UID: "358c8476-1c0d-4bb9-9478-b1e1a43a2a27"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:23:09.230413 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.230387 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34cf95af-4e4e-48e5-84f2-d5bc47be230b-kube-api-access-q9kp8" (OuterVolumeSpecName: "kube-api-access-q9kp8") pod "34cf95af-4e4e-48e5-84f2-d5bc47be230b" (UID: "34cf95af-4e4e-48e5-84f2-d5bc47be230b"). InnerVolumeSpecName "kube-api-access-q9kp8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:23:09.230571 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.230548 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-kube-api-access-vlfqt" (OuterVolumeSpecName: "kube-api-access-vlfqt") pod "358c8476-1c0d-4bb9-9478-b1e1a43a2a27" (UID: "358c8476-1c0d-4bb9-9478-b1e1a43a2a27"). InnerVolumeSpecName "kube-api-access-vlfqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:23:09.329981 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.329950 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9kp8\" (UniqueName: \"kubernetes.io/projected/34cf95af-4e4e-48e5-84f2-d5bc47be230b-kube-api-access-q9kp8\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:23:09.329981 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.329981 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-extensions-socket-volume\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:23:09.330207 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.329992 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlfqt\" (UniqueName: \"kubernetes.io/projected/358c8476-1c0d-4bb9-9478-b1e1a43a2a27-kube-api-access-vlfqt\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:23:09.404251 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.404223 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq"] Jul 07 11:23:09.406378 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.406335 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb"] Jul 07 11:23:09.406515 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:23:09.406335 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc18cde74_7cd5_4bac_9fd5_95ad639e84e7.slice/crio-c7130710065cd5812be4d1c14c83e9372695d4d64bbed368a83b2a59252d07f9 WatchSource:0}: Error finding container c7130710065cd5812be4d1c14c83e9372695d4d64bbed368a83b2a59252d07f9: Status 404 returned error can't find the container with id c7130710065cd5812be4d1c14c83e9372695d4d64bbed368a83b2a59252d07f9 Jul 07 11:23:09.406998 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:23:09.406978 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55d8472a_c726_416d_b095_8658db856f36.slice/crio-0461085ffcf6dc6a73cc8f57e8dd88fa209f821b044ebecff36e70dc2f75a553 WatchSource:0}: Error finding container 0461085ffcf6dc6a73cc8f57e8dd88fa209f821b044ebecff36e70dc2f75a553: Status 404 returned error can't find the container with id 0461085ffcf6dc6a73cc8f57e8dd88fa209f821b044ebecff36e70dc2f75a553 Jul 07 11:23:09.675360 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.675320 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34cf95af-4e4e-48e5-84f2-d5bc47be230b" path="/var/lib/kubelet/pods/34cf95af-4e4e-48e5-84f2-d5bc47be230b/volumes" Jul 07 11:23:09.675732 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.675716 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="358c8476-1c0d-4bb9-9478-b1e1a43a2a27" path="/var/lib/kubelet/pods/358c8476-1c0d-4bb9-9478-b1e1a43a2a27/volumes" Jul 07 11:23:09.826476 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.826445 2585 generic.go:356] "Generic (PLEG): container finished" podID="358c8476-1c0d-4bb9-9478-b1e1a43a2a27" containerID="623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703" exitCode=0 Jul 07 11:23:09.826913 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.826495 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gzjns" Jul 07 11:23:09.826913 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.826555 2585 scope.go:117] "RemoveContainer" containerID="623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703" Jul 07 11:23:09.828499 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.828472 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" event={"ID":"55d8472a-c726-416d-b095-8658db856f36","Type":"ContainerStarted","Data":"572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c"} Jul 07 11:23:09.828629 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.828529 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" event={"ID":"55d8472a-c726-416d-b095-8658db856f36","Type":"ContainerStarted","Data":"0461085ffcf6dc6a73cc8f57e8dd88fa209f821b044ebecff36e70dc2f75a553"} Jul 07 11:23:09.828629 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.828580 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:09.829990 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.829970 2585 generic.go:356] "Generic (PLEG): container finished" podID="34cf95af-4e4e-48e5-84f2-d5bc47be230b" containerID="463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f" exitCode=0 Jul 07 11:23:09.830116 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.830005 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2kkxx" Jul 07 11:23:09.832044 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.832020 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb" event={"ID":"c18cde74-7cd5-4bac-9fd5-95ad639e84e7","Type":"ContainerStarted","Data":"f012035d208e0610c4551450fdeafe3948508cecd8dd49b55cea9cb234dfd18b"} Jul 07 11:23:09.832150 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.832071 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb" event={"ID":"c18cde74-7cd5-4bac-9fd5-95ad639e84e7","Type":"ContainerStarted","Data":"c7130710065cd5812be4d1c14c83e9372695d4d64bbed368a83b2a59252d07f9"} Jul 07 11:23:09.832150 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.832103 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb" Jul 07 11:23:09.839936 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.839917 2585 scope.go:117] "RemoveContainer" containerID="623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703" Jul 07 11:23:09.840242 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:23:09.840222 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703\": container with ID starting with 623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703 not found: ID does not exist" containerID="623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703" Jul 07 11:23:09.840290 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.840254 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703"} err="failed to get container status \"623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703\": rpc error: code = NotFound desc = could not find container \"623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703\": container with ID starting with 623c1231c4c02a0a5cf99eb24c07caffefd1b805fd3ca9479eb8c8ddd74cb703 not found: ID does not exist" Jul 07 11:23:09.840290 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.840271 2585 scope.go:117] "RemoveContainer" containerID="463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f" Jul 07 11:23:09.848629 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.848603 2585 scope.go:117] "RemoveContainer" containerID="463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f" Jul 07 11:23:09.848898 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:23:09.848878 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f\": container with ID starting with 463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f not found: ID does not exist" containerID="463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f" Jul 07 11:23:09.848965 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.848904 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f"} err="failed to get container status \"463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f\": rpc error: code = NotFound desc = could not find container \"463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f\": container with ID starting with 463d265849f5ccfd7ee362a3e60559943b41c7dc11a0ac1941ccd8d25b9c435f not found: ID does not exist" Jul 07 11:23:09.867149 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.867104 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" podStartSLOduration=1.8670883520000001 podStartE2EDuration="1.867088352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:23:09.865128218 +0000 UTC m=+447.452994705" watchObservedRunningTime="2026-07-07 11:23:09.867088352 +0000 UTC m=+447.454954842" Jul 07 11:23:09.918356 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:09.918288 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb" podStartSLOduration=1.918270856 podStartE2EDuration="1.918270856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:23:09.91651995 +0000 UTC m=+447.504386441" watchObservedRunningTime="2026-07-07 11:23:09.918270856 +0000 UTC m=+447.506137345" Jul 07 11:23:20.842361 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:20.842311 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjclb" Jul 07 11:23:20.842798 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:20.842409 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:40.304230 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:40.304195 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq"] Jul 07 11:23:40.304797 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:40.304468 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" podUID="55d8472a-c726-416d-b095-8658db856f36" containerName="manager" containerID="cri-o://572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c" gracePeriod=10 Jul 07 11:23:40.550101 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:40.550078 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:40.598879 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:40.598801 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55d8472a-c726-416d-b095-8658db856f36-extensions-socket-volume\") pod \"55d8472a-c726-416d-b095-8658db856f36\" (UID: \"55d8472a-c726-416d-b095-8658db856f36\") " Jul 07 11:23:40.598879 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:40.598832 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdfws\" (UniqueName: \"kubernetes.io/projected/55d8472a-c726-416d-b095-8658db856f36-kube-api-access-cdfws\") pod \"55d8472a-c726-416d-b095-8658db856f36\" (UID: \"55d8472a-c726-416d-b095-8658db856f36\") " Jul 07 11:23:40.599289 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:40.599262 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55d8472a-c726-416d-b095-8658db856f36-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "55d8472a-c726-416d-b095-8658db856f36" (UID: "55d8472a-c726-416d-b095-8658db856f36"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:23:40.600991 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:40.600952 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55d8472a-c726-416d-b095-8658db856f36-kube-api-access-cdfws" (OuterVolumeSpecName: "kube-api-access-cdfws") pod "55d8472a-c726-416d-b095-8658db856f36" (UID: "55d8472a-c726-416d-b095-8658db856f36"). InnerVolumeSpecName "kube-api-access-cdfws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:23:40.700246 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:40.700209 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55d8472a-c726-416d-b095-8658db856f36-extensions-socket-volume\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:23:40.700246 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:40.700239 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdfws\" (UniqueName: \"kubernetes.io/projected/55d8472a-c726-416d-b095-8658db856f36-kube-api-access-cdfws\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:23:41.001809 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:41.001779 2585 generic.go:356] "Generic (PLEG): container finished" podID="55d8472a-c726-416d-b095-8658db856f36" containerID="572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c" exitCode=0 Jul 07 11:23:41.001959 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:41.001818 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" Jul 07 11:23:41.001959 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:41.001866 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" event={"ID":"55d8472a-c726-416d-b095-8658db856f36","Type":"ContainerDied","Data":"572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c"} Jul 07 11:23:41.001959 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:41.001915 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq" event={"ID":"55d8472a-c726-416d-b095-8658db856f36","Type":"ContainerDied","Data":"0461085ffcf6dc6a73cc8f57e8dd88fa209f821b044ebecff36e70dc2f75a553"} Jul 07 11:23:41.001959 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:41.001937 2585 scope.go:117] "RemoveContainer" containerID="572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c" Jul 07 11:23:41.011291 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:41.011277 2585 scope.go:117] "RemoveContainer" containerID="572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c" Jul 07 11:23:41.011543 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:23:41.011527 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c\": container with ID starting with 572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c not found: ID does not exist" containerID="572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c" Jul 07 11:23:41.011595 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:41.011552 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c"} err="failed to get container status \"572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c\": rpc error: code = NotFound desc = could not find container \"572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c\": container with ID starting with 572e1d1a8f9aff427f36f5a5f95a65a755ba8d5f0d6ad07a33eb6c7436c6c22c not found: ID does not exist" Jul 07 11:23:41.032656 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:41.032632 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq"] Jul 07 11:23:41.040079 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:41.040058 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69xpq"] Jul 07 11:23:41.675922 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:41.675892 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55d8472a-c726-416d-b095-8658db856f36" path="/var/lib/kubelet/pods/55d8472a-c726-416d-b095-8658db856f36/volumes" Jul 07 11:23:56.629601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.629565 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd"] Jul 07 11:23:56.630146 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.630116 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d8472a-c726-416d-b095-8658db856f36" containerName="manager" Jul 07 11:23:56.630146 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.630135 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d8472a-c726-416d-b095-8658db856f36" containerName="manager" Jul 07 11:23:56.630264 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.630256 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="55d8472a-c726-416d-b095-8658db856f36" containerName="manager" Jul 07 11:23:56.633203 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.633182 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.636903 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.636882 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 11:23:56.637022 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.636882 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4s8m9\"" Jul 07 11:23:56.649610 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.649587 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd"] Jul 07 11:23:56.738285 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.738250 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.738494 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.738302 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.738494 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.738416 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.738494 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.738432 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.738494 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.738455 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.738494 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.738492 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.738681 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.738512 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.738681 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.738579 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.738681 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.738635 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rfw7\" (UniqueName: \"kubernetes.io/projected/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-kube-api-access-5rfw7\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.839526 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.839485 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.839736 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.839535 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.839736 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.839565 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.839736 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.839683 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.839736 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.839727 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.839898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.839846 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.839898 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.839883 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.839997 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.839944 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.839997 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.839959 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.840095 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.839993 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rfw7\" (UniqueName: \"kubernetes.io/projected/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-kube-api-access-5rfw7\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.840212 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.840188 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.840308 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.840228 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.840385 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.840312 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.840385 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.840314 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.841927 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.841895 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.842135 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.842118 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.848328 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.848306 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.848651 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.848631 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rfw7\" (UniqueName: \"kubernetes.io/projected/d5601dd8-8e67-4426-bdbf-cf49dfb965e1-kube-api-access-5rfw7\") pod \"maas-default-gateway-openshift-default-56654dcf7-wjnfd\" (UID: \"d5601dd8-8e67-4426-bdbf-cf49dfb965e1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:56.943314 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:56.943227 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:23:57.082325 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:57.082302 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd"] Jul 07 11:23:57.084605 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:23:57.084565 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5601dd8_8e67_4426_bdbf_cf49dfb965e1.slice/crio-9ee2defecbe44c4714ad53764006dbff10d3006e990916dd4bafc28715b6d189 WatchSource:0}: Error finding container 9ee2defecbe44c4714ad53764006dbff10d3006e990916dd4bafc28715b6d189: Status 404 returned error can't find the container with id 9ee2defecbe44c4714ad53764006dbff10d3006e990916dd4bafc28715b6d189 Jul 07 11:23:58.094312 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:58.094277 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" event={"ID":"d5601dd8-8e67-4426-bdbf-cf49dfb965e1","Type":"ContainerStarted","Data":"9ee2defecbe44c4714ad53764006dbff10d3006e990916dd4bafc28715b6d189"} Jul 07 11:23:59.359557 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:59.359520 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 11:23:59.359822 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:59.359596 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 11:23:59.359822 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:23:59.359631 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 11:24:00.107671 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:00.107634 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" event={"ID":"d5601dd8-8e67-4426-bdbf-cf49dfb965e1","Type":"ContainerStarted","Data":"a6f0e9ce1481e46365291e2eac7751136f5849e33dd8e1994163509a63ff3763"} Jul 07 11:24:00.128965 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:00.128912 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" podStartSLOduration=1.856263896 podStartE2EDuration="4.128895021s" totalImagesPullingTime="2.272631125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:23:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:24:00.128301556 +0000 UTC m=+497.716168047" watchObservedRunningTime="2026-07-07 11:24:00.128895021 +0000 UTC m=+497.716761514" Jul 07 11:24:00.944105 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:00.944072 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:24:00.948744 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:00.948722 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:24:01.112968 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.112913 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:24:01.113840 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.113811 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:24:01.168511 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.168479 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-258qk"] Jul 07 11:24:01.170994 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.170977 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:01.173702 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.173677 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 11:24:01.173827 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.173687 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-w7m48\"" Jul 07 11:24:01.184049 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.184020 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-258qk"] Jul 07 11:24:01.264473 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.264388 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-258qk"] Jul 07 11:24:01.285275 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.285245 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cb6c47e8-977b-40a6-b7db-389366f7ee5a-config-file\") pod \"limitador-limitador-64c94fd64f-258qk\" (UID: \"cb6c47e8-977b-40a6-b7db-389366f7ee5a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:01.285468 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.285371 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkscd\" (UniqueName: \"kubernetes.io/projected/cb6c47e8-977b-40a6-b7db-389366f7ee5a-kube-api-access-vkscd\") pod \"limitador-limitador-64c94fd64f-258qk\" (UID: \"cb6c47e8-977b-40a6-b7db-389366f7ee5a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:01.386265 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.386223 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cb6c47e8-977b-40a6-b7db-389366f7ee5a-config-file\") pod \"limitador-limitador-64c94fd64f-258qk\" (UID: \"cb6c47e8-977b-40a6-b7db-389366f7ee5a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:01.386461 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.386272 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkscd\" (UniqueName: \"kubernetes.io/projected/cb6c47e8-977b-40a6-b7db-389366f7ee5a-kube-api-access-vkscd\") pod \"limitador-limitador-64c94fd64f-258qk\" (UID: \"cb6c47e8-977b-40a6-b7db-389366f7ee5a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:01.386884 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.386863 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cb6c47e8-977b-40a6-b7db-389366f7ee5a-config-file\") pod \"limitador-limitador-64c94fd64f-258qk\" (UID: \"cb6c47e8-977b-40a6-b7db-389366f7ee5a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:01.397140 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.397111 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkscd\" (UniqueName: \"kubernetes.io/projected/cb6c47e8-977b-40a6-b7db-389366f7ee5a-kube-api-access-vkscd\") pod \"limitador-limitador-64c94fd64f-258qk\" (UID: \"cb6c47e8-977b-40a6-b7db-389366f7ee5a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:01.486830 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.486792 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:01.628522 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:01.628493 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-258qk"] Jul 07 11:24:01.630264 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:24:01.630236 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb6c47e8_977b_40a6_b7db_389366f7ee5a.slice/crio-d2f21caf818b91f40d67e9226b596e24143164ed6c58a9c20aaf06cf7bea20ce WatchSource:0}: Error finding container d2f21caf818b91f40d67e9226b596e24143164ed6c58a9c20aaf06cf7bea20ce: Status 404 returned error can't find the container with id d2f21caf818b91f40d67e9226b596e24143164ed6c58a9c20aaf06cf7bea20ce Jul 07 11:24:02.015858 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.015809 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-nsgd8"] Jul 07 11:24:02.019494 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.019471 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nsgd8" Jul 07 11:24:02.021981 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.021964 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ddwzz\"" Jul 07 11:24:02.033544 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.033517 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-nsgd8"] Jul 07 11:24:02.119002 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.118966 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" event={"ID":"cb6c47e8-977b-40a6-b7db-389366f7ee5a","Type":"ContainerStarted","Data":"d2f21caf818b91f40d67e9226b596e24143164ed6c58a9c20aaf06cf7bea20ce"} Jul 07 11:24:02.185702 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.185666 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-5s84q"] Jul 07 11:24:02.188068 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.188052 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5s84q" Jul 07 11:24:02.193793 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.193767 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqf4s\" (UniqueName: \"kubernetes.io/projected/b88aa9e9-f9d9-45c4-b085-e924a34fe132-kube-api-access-hqf4s\") pod \"authorino-58cd955675-nsgd8\" (UID: \"b88aa9e9-f9d9-45c4-b085-e924a34fe132\") " pod="kuadrant-system/authorino-58cd955675-nsgd8" Jul 07 11:24:02.198377 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.198339 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5s84q"] Jul 07 11:24:02.294920 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.294824 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2487\" (UniqueName: \"kubernetes.io/projected/61d6a51b-5663-4e43-a968-1830c2bd8a99-kube-api-access-b2487\") pod \"authorino-75d556bd77-5s84q\" (UID: \"61d6a51b-5663-4e43-a968-1830c2bd8a99\") " pod="kuadrant-system/authorino-75d556bd77-5s84q" Jul 07 11:24:02.294920 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.294894 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqf4s\" (UniqueName: \"kubernetes.io/projected/b88aa9e9-f9d9-45c4-b085-e924a34fe132-kube-api-access-hqf4s\") pod \"authorino-58cd955675-nsgd8\" (UID: \"b88aa9e9-f9d9-45c4-b085-e924a34fe132\") " pod="kuadrant-system/authorino-58cd955675-nsgd8" Jul 07 11:24:02.304126 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.304099 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqf4s\" (UniqueName: \"kubernetes.io/projected/b88aa9e9-f9d9-45c4-b085-e924a34fe132-kube-api-access-hqf4s\") pod \"authorino-58cd955675-nsgd8\" (UID: \"b88aa9e9-f9d9-45c4-b085-e924a34fe132\") " pod="kuadrant-system/authorino-58cd955675-nsgd8" Jul 07 11:24:02.329001 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.328976 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nsgd8" Jul 07 11:24:02.399818 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.399781 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2487\" (UniqueName: \"kubernetes.io/projected/61d6a51b-5663-4e43-a968-1830c2bd8a99-kube-api-access-b2487\") pod \"authorino-75d556bd77-5s84q\" (UID: \"61d6a51b-5663-4e43-a968-1830c2bd8a99\") " pod="kuadrant-system/authorino-75d556bd77-5s84q" Jul 07 11:24:02.414392 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.414073 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2487\" (UniqueName: \"kubernetes.io/projected/61d6a51b-5663-4e43-a968-1830c2bd8a99-kube-api-access-b2487\") pod \"authorino-75d556bd77-5s84q\" (UID: \"61d6a51b-5663-4e43-a968-1830c2bd8a99\") " pod="kuadrant-system/authorino-75d556bd77-5s84q" Jul 07 11:24:02.495249 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.495142 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-nsgd8"] Jul 07 11:24:02.496688 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:24:02.496659 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb88aa9e9_f9d9_45c4_b085_e924a34fe132.slice/crio-bc8c5203e564ad4d213c94b9027178b3af96f18cdea94fcd94e7484762d6f5e0 WatchSource:0}: Error finding container bc8c5203e564ad4d213c94b9027178b3af96f18cdea94fcd94e7484762d6f5e0: Status 404 returned error can't find the container with id bc8c5203e564ad4d213c94b9027178b3af96f18cdea94fcd94e7484762d6f5e0 Jul 07 11:24:02.497769 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.497733 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5s84q" Jul 07 11:24:02.695219 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:02.695184 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5s84q"] Jul 07 11:24:02.700798 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:24:02.700732 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61d6a51b_5663_4e43_a968_1830c2bd8a99.slice/crio-76ae56f5ee28367c09309689fc00503ec54d0c7aa5e0a7689853686ec7bbfe90 WatchSource:0}: Error finding container 76ae56f5ee28367c09309689fc00503ec54d0c7aa5e0a7689853686ec7bbfe90: Status 404 returned error can't find the container with id 76ae56f5ee28367c09309689fc00503ec54d0c7aa5e0a7689853686ec7bbfe90 Jul 07 11:24:03.127733 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:03.127691 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5s84q" event={"ID":"61d6a51b-5663-4e43-a968-1830c2bd8a99","Type":"ContainerStarted","Data":"76ae56f5ee28367c09309689fc00503ec54d0c7aa5e0a7689853686ec7bbfe90"} Jul 07 11:24:03.130068 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:03.130020 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nsgd8" event={"ID":"b88aa9e9-f9d9-45c4-b085-e924a34fe132","Type":"ContainerStarted","Data":"bc8c5203e564ad4d213c94b9027178b3af96f18cdea94fcd94e7484762d6f5e0"} Jul 07 11:24:07.165544 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:07.165510 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nsgd8" event={"ID":"b88aa9e9-f9d9-45c4-b085-e924a34fe132","Type":"ContainerStarted","Data":"ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62"} Jul 07 11:24:07.167115 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:07.167086 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" event={"ID":"cb6c47e8-977b-40a6-b7db-389366f7ee5a","Type":"ContainerStarted","Data":"925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615"} Jul 07 11:24:07.167245 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:07.167172 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:07.168464 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:07.168442 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5s84q" event={"ID":"61d6a51b-5663-4e43-a968-1830c2bd8a99","Type":"ContainerStarted","Data":"1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0"} Jul 07 11:24:07.181357 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:07.181297 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-nsgd8" podStartSLOduration=1.7066414939999999 podStartE2EDuration="6.181281262s" totalImagesPullingTime="4.474639768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:24:07.180692768 +0000 UTC m=+504.768559259" watchObservedRunningTime="2026-07-07 11:24:07.181281262 +0000 UTC m=+504.769147753" Jul 07 11:24:07.203929 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:07.203864 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" podStartSLOduration=0.850078354 podStartE2EDuration="6.20384083s" totalImagesPullingTime="5.353762476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:24:07.199832766 +0000 UTC m=+504.787699257" watchObservedRunningTime="2026-07-07 11:24:07.20384083 +0000 UTC m=+504.791707326" Jul 07 11:24:07.216989 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:07.216933 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-5s84q" podStartSLOduration=0.97562636 podStartE2EDuration="5.216915751s" totalImagesPullingTime="4.241289391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:24:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:24:07.214644095 +0000 UTC m=+504.802510601" watchObservedRunningTime="2026-07-07 11:24:07.216915751 +0000 UTC m=+504.804782240" Jul 07 11:24:07.241188 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:07.241152 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-nsgd8"] Jul 07 11:24:09.179154 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:09.179111 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-nsgd8" podUID="b88aa9e9-f9d9-45c4-b085-e924a34fe132" containerName="authorino" containerID="cri-o://ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62" gracePeriod=30 Jul 07 11:24:09.424678 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:09.424653 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nsgd8" Jul 07 11:24:09.459713 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:09.459642 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqf4s\" (UniqueName: \"kubernetes.io/projected/b88aa9e9-f9d9-45c4-b085-e924a34fe132-kube-api-access-hqf4s\") pod \"b88aa9e9-f9d9-45c4-b085-e924a34fe132\" (UID: \"b88aa9e9-f9d9-45c4-b085-e924a34fe132\") " Jul 07 11:24:09.461622 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:09.461595 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b88aa9e9-f9d9-45c4-b085-e924a34fe132-kube-api-access-hqf4s" (OuterVolumeSpecName: "kube-api-access-hqf4s") pod "b88aa9e9-f9d9-45c4-b085-e924a34fe132" (UID: "b88aa9e9-f9d9-45c4-b085-e924a34fe132"). InnerVolumeSpecName "kube-api-access-hqf4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:24:09.561110 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:09.561067 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqf4s\" (UniqueName: \"kubernetes.io/projected/b88aa9e9-f9d9-45c4-b085-e924a34fe132-kube-api-access-hqf4s\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:24:10.185495 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:10.185458 2585 generic.go:356] "Generic (PLEG): container finished" podID="b88aa9e9-f9d9-45c4-b085-e924a34fe132" containerID="ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62" exitCode=0 Jul 07 11:24:10.185935 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:10.185502 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nsgd8" Jul 07 11:24:10.185935 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:10.185539 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nsgd8" event={"ID":"b88aa9e9-f9d9-45c4-b085-e924a34fe132","Type":"ContainerDied","Data":"ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62"} Jul 07 11:24:10.185935 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:10.185579 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nsgd8" event={"ID":"b88aa9e9-f9d9-45c4-b085-e924a34fe132","Type":"ContainerDied","Data":"bc8c5203e564ad4d213c94b9027178b3af96f18cdea94fcd94e7484762d6f5e0"} Jul 07 11:24:10.185935 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:10.185596 2585 scope.go:117] "RemoveContainer" containerID="ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62" Jul 07 11:24:10.195008 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:10.194989 2585 scope.go:117] "RemoveContainer" containerID="ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62" Jul 07 11:24:10.195389 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:24:10.195367 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62\": container with ID starting with ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62 not found: ID does not exist" containerID="ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62" Jul 07 11:24:10.195477 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:10.195398 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62"} err="failed to get container status \"ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62\": rpc error: code = NotFound desc = could not find container \"ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62\": container with ID starting with ed4da9061e2a1db3b953374c10830889212cb8b648cad13890a097aa13e51f62 not found: ID does not exist" Jul 07 11:24:10.205216 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:10.205193 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-nsgd8"] Jul 07 11:24:10.208821 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:10.208799 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-nsgd8"] Jul 07 11:24:11.676049 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:11.676016 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b88aa9e9-f9d9-45c4-b085-e924a34fe132" path="/var/lib/kubelet/pods/b88aa9e9-f9d9-45c4-b085-e924a34fe132/volumes" Jul 07 11:24:16.455903 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:16.455862 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-258qk"] Jul 07 11:24:16.456400 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:16.456149 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" podUID="cb6c47e8-977b-40a6-b7db-389366f7ee5a" containerName="limitador" containerID="cri-o://925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615" gracePeriod=30 Jul 07 11:24:16.458681 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:16.458660 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:17.007162 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.007138 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:17.016274 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.016219 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cb6c47e8-977b-40a6-b7db-389366f7ee5a-config-file\") pod \"cb6c47e8-977b-40a6-b7db-389366f7ee5a\" (UID: \"cb6c47e8-977b-40a6-b7db-389366f7ee5a\") " Jul 07 11:24:17.016274 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.016271 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkscd\" (UniqueName: \"kubernetes.io/projected/cb6c47e8-977b-40a6-b7db-389366f7ee5a-kube-api-access-vkscd\") pod \"cb6c47e8-977b-40a6-b7db-389366f7ee5a\" (UID: \"cb6c47e8-977b-40a6-b7db-389366f7ee5a\") " Jul 07 11:24:17.016617 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.016596 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb6c47e8-977b-40a6-b7db-389366f7ee5a-config-file" (OuterVolumeSpecName: "config-file") pod "cb6c47e8-977b-40a6-b7db-389366f7ee5a" (UID: "cb6c47e8-977b-40a6-b7db-389366f7ee5a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:24:17.018268 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.018246 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb6c47e8-977b-40a6-b7db-389366f7ee5a-kube-api-access-vkscd" (OuterVolumeSpecName: "kube-api-access-vkscd") pod "cb6c47e8-977b-40a6-b7db-389366f7ee5a" (UID: "cb6c47e8-977b-40a6-b7db-389366f7ee5a"). InnerVolumeSpecName "kube-api-access-vkscd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:24:17.116866 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.116830 2585 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cb6c47e8-977b-40a6-b7db-389366f7ee5a-config-file\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:24:17.116866 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.116859 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkscd\" (UniqueName: \"kubernetes.io/projected/cb6c47e8-977b-40a6-b7db-389366f7ee5a-kube-api-access-vkscd\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:24:17.223128 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.223097 2585 generic.go:356] "Generic (PLEG): container finished" podID="cb6c47e8-977b-40a6-b7db-389366f7ee5a" containerID="925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615" exitCode=0 Jul 07 11:24:17.223291 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.223138 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" Jul 07 11:24:17.223291 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.223156 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" event={"ID":"cb6c47e8-977b-40a6-b7db-389366f7ee5a","Type":"ContainerDied","Data":"925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615"} Jul 07 11:24:17.223291 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.223197 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-258qk" event={"ID":"cb6c47e8-977b-40a6-b7db-389366f7ee5a","Type":"ContainerDied","Data":"d2f21caf818b91f40d67e9226b596e24143164ed6c58a9c20aaf06cf7bea20ce"} Jul 07 11:24:17.223291 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.223218 2585 scope.go:117] "RemoveContainer" containerID="925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615" Jul 07 11:24:17.232118 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.232101 2585 scope.go:117] "RemoveContainer" containerID="925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615" Jul 07 11:24:17.232385 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:24:17.232366 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615\": container with ID starting with 925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615 not found: ID does not exist" containerID="925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615" Jul 07 11:24:17.232429 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.232394 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615"} err="failed to get container status \"925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615\": rpc error: code = NotFound desc = could not find container \"925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615\": container with ID starting with 925f30a97fd154b88bea11215024b20fa4f6a05622e4c2131ca1f632e91d0615 not found: ID does not exist" Jul 07 11:24:17.248548 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.248520 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-258qk"] Jul 07 11:24:17.251439 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.251419 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-258qk"] Jul 07 11:24:17.675230 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:17.675196 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb6c47e8-977b-40a6-b7db-389366f7ee5a" path="/var/lib/kubelet/pods/cb6c47e8-977b-40a6-b7db-389366f7ee5a/volumes" Jul 07 11:24:22.682650 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.682619 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-tpxql"] Jul 07 11:24:22.683024 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.683010 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb6c47e8-977b-40a6-b7db-389366f7ee5a" containerName="limitador" Jul 07 11:24:22.683063 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.683025 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb6c47e8-977b-40a6-b7db-389366f7ee5a" containerName="limitador" Jul 07 11:24:22.683063 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.683045 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b88aa9e9-f9d9-45c4-b085-e924a34fe132" containerName="authorino" Jul 07 11:24:22.683063 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.683050 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b88aa9e9-f9d9-45c4-b085-e924a34fe132" containerName="authorino" Jul 07 11:24:22.683158 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.683105 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb6c47e8-977b-40a6-b7db-389366f7ee5a" containerName="limitador" Jul 07 11:24:22.683158 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.683112 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b88aa9e9-f9d9-45c4-b085-e924a34fe132" containerName="authorino" Jul 07 11:24:22.685936 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.685905 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-tpxql" Jul 07 11:24:22.689178 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.689135 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 11:24:22.689178 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.689138 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-c2dgz\"" Jul 07 11:24:22.696327 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.696302 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-tpxql"] Jul 07 11:24:22.755651 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.755617 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/342a50aa-58ff-4643-add7-01939a5bb888-data\") pod \"postgres-69fb4594cf-tpxql\" (UID: \"342a50aa-58ff-4643-add7-01939a5bb888\") " pod="opendatahub/postgres-69fb4594cf-tpxql" Jul 07 11:24:22.755797 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.755683 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brzjp\" (UniqueName: \"kubernetes.io/projected/342a50aa-58ff-4643-add7-01939a5bb888-kube-api-access-brzjp\") pod \"postgres-69fb4594cf-tpxql\" (UID: \"342a50aa-58ff-4643-add7-01939a5bb888\") " pod="opendatahub/postgres-69fb4594cf-tpxql" Jul 07 11:24:22.856475 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.856442 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brzjp\" (UniqueName: \"kubernetes.io/projected/342a50aa-58ff-4643-add7-01939a5bb888-kube-api-access-brzjp\") pod \"postgres-69fb4594cf-tpxql\" (UID: \"342a50aa-58ff-4643-add7-01939a5bb888\") " pod="opendatahub/postgres-69fb4594cf-tpxql" Jul 07 11:24:22.856663 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.856523 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/342a50aa-58ff-4643-add7-01939a5bb888-data\") pod \"postgres-69fb4594cf-tpxql\" (UID: \"342a50aa-58ff-4643-add7-01939a5bb888\") " pod="opendatahub/postgres-69fb4594cf-tpxql" Jul 07 11:24:22.856876 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.856858 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/342a50aa-58ff-4643-add7-01939a5bb888-data\") pod \"postgres-69fb4594cf-tpxql\" (UID: \"342a50aa-58ff-4643-add7-01939a5bb888\") " pod="opendatahub/postgres-69fb4594cf-tpxql" Jul 07 11:24:22.865808 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.865786 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brzjp\" (UniqueName: \"kubernetes.io/projected/342a50aa-58ff-4643-add7-01939a5bb888-kube-api-access-brzjp\") pod \"postgres-69fb4594cf-tpxql\" (UID: \"342a50aa-58ff-4643-add7-01939a5bb888\") " pod="opendatahub/postgres-69fb4594cf-tpxql" Jul 07 11:24:23.000048 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:22.999948 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-tpxql" Jul 07 11:24:23.130766 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:23.130737 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-tpxql"] Jul 07 11:24:23.132628 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:24:23.132592 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod342a50aa_58ff_4643_add7_01939a5bb888.slice/crio-a4a8b77cb54442115fe9c96001ad68d99bb2ffd45ca8c3c5d8f1f2852d404c6c WatchSource:0}: Error finding container a4a8b77cb54442115fe9c96001ad68d99bb2ffd45ca8c3c5d8f1f2852d404c6c: Status 404 returned error can't find the container with id a4a8b77cb54442115fe9c96001ad68d99bb2ffd45ca8c3c5d8f1f2852d404c6c Jul 07 11:24:23.256134 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:23.256043 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-tpxql" event={"ID":"342a50aa-58ff-4643-add7-01939a5bb888","Type":"ContainerStarted","Data":"a4a8b77cb54442115fe9c96001ad68d99bb2ffd45ca8c3c5d8f1f2852d404c6c"} Jul 07 11:24:28.291817 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:28.291772 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-tpxql" event={"ID":"342a50aa-58ff-4643-add7-01939a5bb888","Type":"ContainerStarted","Data":"bad7b5c1c4ce906e43470d1c3b27e3b22f712778dd04b7e6eab28e02538eca99"} Jul 07 11:24:28.292194 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:28.291901 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-tpxql" Jul 07 11:24:28.308533 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:28.308484 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-tpxql" podStartSLOduration=1.301686695 podStartE2EDuration="6.308470478s" totalImagesPullingTime="5.006783783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:24:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:24:28.306647286 +0000 UTC m=+525.894513772" watchObservedRunningTime="2026-07-07 11:24:28.308470478 +0000 UTC m=+525.896337002" Jul 07 11:24:34.327550 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:34.327471 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-tpxql" Jul 07 11:24:34.889110 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:34.889078 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhltf"] Jul 07 11:24:34.899002 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:34.898973 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhltf" Jul 07 11:24:34.899529 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:34.899503 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhltf"] Jul 07 11:24:34.954088 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:34.954052 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pjs8\" (UniqueName: \"kubernetes.io/projected/4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c-kube-api-access-8pjs8\") pod \"authorino-5d4b67c446-mhltf\" (UID: \"4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c\") " pod="kuadrant-system/authorino-5d4b67c446-mhltf" Jul 07 11:24:35.055419 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.055375 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pjs8\" (UniqueName: \"kubernetes.io/projected/4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c-kube-api-access-8pjs8\") pod \"authorino-5d4b67c446-mhltf\" (UID: \"4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c\") " pod="kuadrant-system/authorino-5d4b67c446-mhltf" Jul 07 11:24:35.065011 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.064986 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pjs8\" (UniqueName: \"kubernetes.io/projected/4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c-kube-api-access-8pjs8\") pod \"authorino-5d4b67c446-mhltf\" (UID: \"4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c\") " pod="kuadrant-system/authorino-5d4b67c446-mhltf" Jul 07 11:24:35.140775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.140680 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhltf"] Jul 07 11:24:35.140977 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.140964 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhltf" Jul 07 11:24:35.213720 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.213692 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7dc9f95f4c-kjdkw"] Jul 07 11:24:35.218206 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.218189 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dc9f95f4c-kjdkw" Jul 07 11:24:35.231677 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.231650 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dc9f95f4c-kjdkw"] Jul 07 11:24:35.257533 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.257505 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs5cj\" (UniqueName: \"kubernetes.io/projected/424b46e8-2484-411a-9841-586ec4679f06-kube-api-access-xs5cj\") pod \"authorino-7dc9f95f4c-kjdkw\" (UID: \"424b46e8-2484-411a-9841-586ec4679f06\") " pod="kuadrant-system/authorino-7dc9f95f4c-kjdkw" Jul 07 11:24:35.285484 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.285459 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhltf"] Jul 07 11:24:35.286469 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:24:35.286449 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ad92abc_7dcf_4786_b13c_bd8a0ac7e37c.slice/crio-770d65d5abf59f26b8de9883eeaf2e293b04f62f297af502d380fdffa9c47dd3 WatchSource:0}: Error finding container 770d65d5abf59f26b8de9883eeaf2e293b04f62f297af502d380fdffa9c47dd3: Status 404 returned error can't find the container with id 770d65d5abf59f26b8de9883eeaf2e293b04f62f297af502d380fdffa9c47dd3 Jul 07 11:24:35.329756 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.329727 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mhltf" event={"ID":"4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c","Type":"ContainerStarted","Data":"770d65d5abf59f26b8de9883eeaf2e293b04f62f297af502d380fdffa9c47dd3"} Jul 07 11:24:35.339648 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.339624 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7dc9f95f4c-kjdkw"] Jul 07 11:24:35.339892 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:24:35.339874 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xs5cj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7dc9f95f4c-kjdkw" podUID="424b46e8-2484-411a-9841-586ec4679f06" Jul 07 11:24:35.358330 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.358304 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs5cj\" (UniqueName: \"kubernetes.io/projected/424b46e8-2484-411a-9841-586ec4679f06-kube-api-access-xs5cj\") pod \"authorino-7dc9f95f4c-kjdkw\" (UID: \"424b46e8-2484-411a-9841-586ec4679f06\") " pod="kuadrant-system/authorino-7dc9f95f4c-kjdkw" Jul 07 11:24:35.373025 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.372996 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs5cj\" (UniqueName: \"kubernetes.io/projected/424b46e8-2484-411a-9841-586ec4679f06-kube-api-access-xs5cj\") pod \"authorino-7dc9f95f4c-kjdkw\" (UID: \"424b46e8-2484-411a-9841-586ec4679f06\") " pod="kuadrant-system/authorino-7dc9f95f4c-kjdkw" Jul 07 11:24:35.381452 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.381432 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-789c94966c-99rtj"] Jul 07 11:24:35.384826 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.384813 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-789c94966c-99rtj" Jul 07 11:24:35.388412 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.388390 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 11:24:35.403101 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.403071 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-789c94966c-99rtj"] Jul 07 11:24:35.458763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.458731 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-tls-cert\") pod \"authorino-789c94966c-99rtj\" (UID: \"68370995-bc5e-45a2-8dc4-84faa4bd6e2f\") " pod="kuadrant-system/authorino-789c94966c-99rtj" Jul 07 11:24:35.458763 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.458765 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tskv\" (UniqueName: \"kubernetes.io/projected/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-kube-api-access-7tskv\") pod \"authorino-789c94966c-99rtj\" (UID: \"68370995-bc5e-45a2-8dc4-84faa4bd6e2f\") " pod="kuadrant-system/authorino-789c94966c-99rtj" Jul 07 11:24:35.560144 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.560105 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-tls-cert\") pod \"authorino-789c94966c-99rtj\" (UID: \"68370995-bc5e-45a2-8dc4-84faa4bd6e2f\") " pod="kuadrant-system/authorino-789c94966c-99rtj" Jul 07 11:24:35.560144 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.560147 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tskv\" (UniqueName: \"kubernetes.io/projected/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-kube-api-access-7tskv\") pod \"authorino-789c94966c-99rtj\" (UID: \"68370995-bc5e-45a2-8dc4-84faa4bd6e2f\") " pod="kuadrant-system/authorino-789c94966c-99rtj" Jul 07 11:24:35.562620 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.562601 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-tls-cert\") pod \"authorino-789c94966c-99rtj\" (UID: \"68370995-bc5e-45a2-8dc4-84faa4bd6e2f\") " pod="kuadrant-system/authorino-789c94966c-99rtj" Jul 07 11:24:35.568072 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.568054 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tskv\" (UniqueName: \"kubernetes.io/projected/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-kube-api-access-7tskv\") pod \"authorino-789c94966c-99rtj\" (UID: \"68370995-bc5e-45a2-8dc4-84faa4bd6e2f\") " pod="kuadrant-system/authorino-789c94966c-99rtj" Jul 07 11:24:35.693452 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.693430 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-789c94966c-99rtj" Jul 07 11:24:35.837400 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:35.837377 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-789c94966c-99rtj"] Jul 07 11:24:35.839734 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:24:35.839709 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68370995_bc5e_45a2_8dc4_84faa4bd6e2f.slice/crio-fb6a0900cddf5f012c689265baad596e10c3ba484b2e6af8cc9ca78cab6db72f WatchSource:0}: Error finding container fb6a0900cddf5f012c689265baad596e10c3ba484b2e6af8cc9ca78cab6db72f: Status 404 returned error can't find the container with id fb6a0900cddf5f012c689265baad596e10c3ba484b2e6af8cc9ca78cab6db72f Jul 07 11:24:36.337327 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.337294 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-789c94966c-99rtj" event={"ID":"68370995-bc5e-45a2-8dc4-84faa4bd6e2f","Type":"ContainerStarted","Data":"0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc"} Jul 07 11:24:36.337847 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.337340 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-789c94966c-99rtj" event={"ID":"68370995-bc5e-45a2-8dc4-84faa4bd6e2f","Type":"ContainerStarted","Data":"fb6a0900cddf5f012c689265baad596e10c3ba484b2e6af8cc9ca78cab6db72f"} Jul 07 11:24:36.339046 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.339016 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mhltf" event={"ID":"4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c","Type":"ContainerStarted","Data":"4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f"} Jul 07 11:24:36.339178 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.339048 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dc9f95f4c-kjdkw" Jul 07 11:24:36.339178 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.339050 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-mhltf" podUID="4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c" containerName="authorino" containerID="cri-o://4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f" gracePeriod=30 Jul 07 11:24:36.346828 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.346809 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dc9f95f4c-kjdkw" Jul 07 11:24:36.355172 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.355119 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-789c94966c-99rtj" podStartSLOduration=0.969528699 podStartE2EDuration="1.355100029s" totalImagesPullingTime="385.57133ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:24:36.354054435 +0000 UTC m=+533.941920942" watchObservedRunningTime="2026-07-07 11:24:36.355100029 +0000 UTC m=+533.942966520" Jul 07 11:24:36.368689 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.368659 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xs5cj\" (UniqueName: \"kubernetes.io/projected/424b46e8-2484-411a-9841-586ec4679f06-kube-api-access-xs5cj\") pod \"424b46e8-2484-411a-9841-586ec4679f06\" (UID: \"424b46e8-2484-411a-9841-586ec4679f06\") " Jul 07 11:24:36.371078 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.371052 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/424b46e8-2484-411a-9841-586ec4679f06-kube-api-access-xs5cj" (OuterVolumeSpecName: "kube-api-access-xs5cj") pod "424b46e8-2484-411a-9841-586ec4679f06" (UID: "424b46e8-2484-411a-9841-586ec4679f06"). InnerVolumeSpecName "kube-api-access-xs5cj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:24:36.381700 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.381642 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-mhltf" podStartSLOduration=2.018846989 podStartE2EDuration="2.381619379s" totalImagesPullingTime="362.77239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:24:36.378284457 +0000 UTC m=+533.966150951" watchObservedRunningTime="2026-07-07 11:24:36.381619379 +0000 UTC m=+533.969485870" Jul 07 11:24:36.392400 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.392141 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5s84q"] Jul 07 11:24:36.392400 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.392332 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-5s84q" podUID="61d6a51b-5663-4e43-a968-1830c2bd8a99" containerName="authorino" containerID="cri-o://1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0" gracePeriod=30 Jul 07 11:24:36.470231 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.470199 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs5cj\" (UniqueName: \"kubernetes.io/projected/424b46e8-2484-411a-9841-586ec4679f06-kube-api-access-xs5cj\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:24:36.640931 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.640903 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhltf" Jul 07 11:24:36.649524 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.649501 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5s84q" Jul 07 11:24:36.671574 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.671544 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2487\" (UniqueName: \"kubernetes.io/projected/61d6a51b-5663-4e43-a968-1830c2bd8a99-kube-api-access-b2487\") pod \"61d6a51b-5663-4e43-a968-1830c2bd8a99\" (UID: \"61d6a51b-5663-4e43-a968-1830c2bd8a99\") " Jul 07 11:24:36.671743 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.671591 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pjs8\" (UniqueName: \"kubernetes.io/projected/4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c-kube-api-access-8pjs8\") pod \"4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c\" (UID: \"4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c\") " Jul 07 11:24:36.673984 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.673955 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61d6a51b-5663-4e43-a968-1830c2bd8a99-kube-api-access-b2487" (OuterVolumeSpecName: "kube-api-access-b2487") pod "61d6a51b-5663-4e43-a968-1830c2bd8a99" (UID: "61d6a51b-5663-4e43-a968-1830c2bd8a99"). InnerVolumeSpecName "kube-api-access-b2487". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:24:36.674118 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.674082 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c-kube-api-access-8pjs8" (OuterVolumeSpecName: "kube-api-access-8pjs8") pod "4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c" (UID: "4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c"). InnerVolumeSpecName "kube-api-access-8pjs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:24:36.773337 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.773238 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2487\" (UniqueName: \"kubernetes.io/projected/61d6a51b-5663-4e43-a968-1830c2bd8a99-kube-api-access-b2487\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:24:36.773337 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:36.773267 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pjs8\" (UniqueName: \"kubernetes.io/projected/4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c-kube-api-access-8pjs8\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:24:37.345503 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.345426 2585 generic.go:356] "Generic (PLEG): container finished" podID="61d6a51b-5663-4e43-a968-1830c2bd8a99" containerID="1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0" exitCode=0 Jul 07 11:24:37.345503 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.345473 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5s84q" Jul 07 11:24:37.345503 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.345493 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5s84q" event={"ID":"61d6a51b-5663-4e43-a968-1830c2bd8a99","Type":"ContainerDied","Data":"1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0"} Jul 07 11:24:37.345991 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.345532 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5s84q" event={"ID":"61d6a51b-5663-4e43-a968-1830c2bd8a99","Type":"ContainerDied","Data":"76ae56f5ee28367c09309689fc00503ec54d0c7aa5e0a7689853686ec7bbfe90"} Jul 07 11:24:37.345991 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.345556 2585 scope.go:117] "RemoveContainer" containerID="1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0" Jul 07 11:24:37.348267 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.348238 2585 generic.go:356] "Generic (PLEG): container finished" podID="4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c" containerID="4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f" exitCode=0 Jul 07 11:24:37.348421 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.348287 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhltf" Jul 07 11:24:37.348421 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.348322 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mhltf" event={"ID":"4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c","Type":"ContainerDied","Data":"4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f"} Jul 07 11:24:37.348546 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.348427 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dc9f95f4c-kjdkw" Jul 07 11:24:37.349010 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.348981 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mhltf" event={"ID":"4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c","Type":"ContainerDied","Data":"770d65d5abf59f26b8de9883eeaf2e293b04f62f297af502d380fdffa9c47dd3"} Jul 07 11:24:37.356695 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.356663 2585 scope.go:117] "RemoveContainer" containerID="1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0" Jul 07 11:24:37.357007 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:24:37.356971 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0\": container with ID starting with 1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0 not found: ID does not exist" containerID="1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0" Jul 07 11:24:37.357115 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.357006 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0"} err="failed to get container status \"1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0\": rpc error: code = NotFound desc = could not find container \"1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0\": container with ID starting with 1d6dde686c231ec1399a233d5499d34241261ba0c72ea2d7e06618a7804058a0 not found: ID does not exist" Jul 07 11:24:37.357115 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.357028 2585 scope.go:117] "RemoveContainer" containerID="4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f" Jul 07 11:24:37.365393 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.365378 2585 scope.go:117] "RemoveContainer" containerID="4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f" Jul 07 11:24:37.365639 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:24:37.365622 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f\": container with ID starting with 4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f not found: ID does not exist" containerID="4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f" Jul 07 11:24:37.365688 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.365647 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f"} err="failed to get container status \"4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f\": rpc error: code = NotFound desc = could not find container \"4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f\": container with ID starting with 4b86984e4ca9aaf63ee460dad9eac36bafd1871e9995ab6c55dca1295ec5892f not found: ID does not exist" Jul 07 11:24:37.381473 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.381417 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7dc9f95f4c-kjdkw"] Jul 07 11:24:37.385210 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.385187 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7dc9f95f4c-kjdkw"] Jul 07 11:24:37.396123 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.396099 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5s84q"] Jul 07 11:24:37.400075 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.400038 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5s84q"] Jul 07 11:24:37.410375 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.410332 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhltf"] Jul 07 11:24:37.413663 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.413639 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhltf"] Jul 07 11:24:37.676055 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.676013 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="424b46e8-2484-411a-9841-586ec4679f06" path="/var/lib/kubelet/pods/424b46e8-2484-411a-9841-586ec4679f06/volumes" Jul 07 11:24:37.676329 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.676312 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c" path="/var/lib/kubelet/pods/4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c/volumes" Jul 07 11:24:37.676718 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:37.676699 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61d6a51b-5663-4e43-a968-1830c2bd8a99" path="/var/lib/kubelet/pods/61d6a51b-5663-4e43-a968-1830c2bd8a99/volumes" Jul 07 11:24:40.629748 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.629712 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5dbc579747-drs6s"] Jul 07 11:24:40.630273 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.630254 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61d6a51b-5663-4e43-a968-1830c2bd8a99" containerName="authorino" Jul 07 11:24:40.630411 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.630276 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="61d6a51b-5663-4e43-a968-1830c2bd8a99" containerName="authorino" Jul 07 11:24:40.630411 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.630308 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c" containerName="authorino" Jul 07 11:24:40.630411 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.630316 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c" containerName="authorino" Jul 07 11:24:40.630570 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.630448 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ad92abc-7dcf-4786-b13c-bd8a0ac7e37c" containerName="authorino" Jul 07 11:24:40.630570 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.630466 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="61d6a51b-5663-4e43-a968-1830c2bd8a99" containerName="authorino" Jul 07 11:24:40.693772 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.693735 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dbc579747-drs6s"] Jul 07 11:24:40.693943 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.693850 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:40.697867 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.697833 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 11:24:40.698026 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.697921 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dxl7v\"" Jul 07 11:24:40.698137 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.698116 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 11:24:40.807406 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.807369 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bf9578fbb-5grln"] Jul 07 11:24:40.809501 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.809479 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a889086f-4bbf-41a1-b314-3008b9c2aad8-webhook-cert\") pod \"maas-controller-5dbc579747-drs6s\" (UID: \"a889086f-4bbf-41a1-b314-3008b9c2aad8\") " pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:40.809658 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.809634 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz5g6\" (UniqueName: \"kubernetes.io/projected/a889086f-4bbf-41a1-b314-3008b9c2aad8-kube-api-access-sz5g6\") pod \"maas-controller-5dbc579747-drs6s\" (UID: \"a889086f-4bbf-41a1-b314-3008b9c2aad8\") " pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:40.828650 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.828626 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bf9578fbb-5grln"] Jul 07 11:24:40.828758 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.828721 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:24:40.910977 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.910896 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz5g6\" (UniqueName: \"kubernetes.io/projected/a889086f-4bbf-41a1-b314-3008b9c2aad8-kube-api-access-sz5g6\") pod \"maas-controller-5dbc579747-drs6s\" (UID: \"a889086f-4bbf-41a1-b314-3008b9c2aad8\") " pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:40.910977 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.910940 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a889086f-4bbf-41a1-b314-3008b9c2aad8-webhook-cert\") pod \"maas-controller-5dbc579747-drs6s\" (UID: \"a889086f-4bbf-41a1-b314-3008b9c2aad8\") " pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:40.910977 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.910968 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b518c36-05fd-48af-a2a3-1571be5cb852-webhook-cert\") pod \"maas-controller-5bf9578fbb-5grln\" (UID: \"9b518c36-05fd-48af-a2a3-1571be5cb852\") " pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:24:40.911247 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:24:40.911076 2585 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 07 11:24:40.911247 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.911110 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcs9s\" (UniqueName: \"kubernetes.io/projected/9b518c36-05fd-48af-a2a3-1571be5cb852-kube-api-access-bcs9s\") pod \"maas-controller-5bf9578fbb-5grln\" (UID: \"9b518c36-05fd-48af-a2a3-1571be5cb852\") " pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:24:40.911247 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:24:40.911146 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a889086f-4bbf-41a1-b314-3008b9c2aad8-webhook-cert podName:a889086f-4bbf-41a1-b314-3008b9c2aad8 nodeName:}" failed. No retries permitted until 2026-07-07 11:24:41.411125807 +0000 UTC m=+538.998992287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/a889086f-4bbf-41a1-b314-3008b9c2aad8-webhook-cert") pod "maas-controller-5dbc579747-drs6s" (UID: "a889086f-4bbf-41a1-b314-3008b9c2aad8") : secret "maas-controller-webhook-cert" not found Jul 07 11:24:40.919749 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:40.919727 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz5g6\" (UniqueName: \"kubernetes.io/projected/a889086f-4bbf-41a1-b314-3008b9c2aad8-kube-api-access-sz5g6\") pod \"maas-controller-5dbc579747-drs6s\" (UID: \"a889086f-4bbf-41a1-b314-3008b9c2aad8\") " pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:41.011910 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:41.011876 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b518c36-05fd-48af-a2a3-1571be5cb852-webhook-cert\") pod \"maas-controller-5bf9578fbb-5grln\" (UID: \"9b518c36-05fd-48af-a2a3-1571be5cb852\") " pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:24:41.012082 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:41.011930 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcs9s\" (UniqueName: \"kubernetes.io/projected/9b518c36-05fd-48af-a2a3-1571be5cb852-kube-api-access-bcs9s\") pod \"maas-controller-5bf9578fbb-5grln\" (UID: \"9b518c36-05fd-48af-a2a3-1571be5cb852\") " pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:24:41.012082 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:24:41.012028 2585 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 07 11:24:41.012161 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:24:41.012090 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b518c36-05fd-48af-a2a3-1571be5cb852-webhook-cert podName:9b518c36-05fd-48af-a2a3-1571be5cb852 nodeName:}" failed. No retries permitted until 2026-07-07 11:24:41.512074712 +0000 UTC m=+539.099941183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9b518c36-05fd-48af-a2a3-1571be5cb852-webhook-cert") pod "maas-controller-5bf9578fbb-5grln" (UID: "9b518c36-05fd-48af-a2a3-1571be5cb852") : secret "maas-controller-webhook-cert" not found Jul 07 11:24:41.020438 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:41.020412 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcs9s\" (UniqueName: \"kubernetes.io/projected/9b518c36-05fd-48af-a2a3-1571be5cb852-kube-api-access-bcs9s\") pod \"maas-controller-5bf9578fbb-5grln\" (UID: \"9b518c36-05fd-48af-a2a3-1571be5cb852\") " pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:24:41.416084 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:41.416042 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a889086f-4bbf-41a1-b314-3008b9c2aad8-webhook-cert\") pod \"maas-controller-5dbc579747-drs6s\" (UID: \"a889086f-4bbf-41a1-b314-3008b9c2aad8\") " pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:41.418508 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:41.418473 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a889086f-4bbf-41a1-b314-3008b9c2aad8-webhook-cert\") pod \"maas-controller-5dbc579747-drs6s\" (UID: \"a889086f-4bbf-41a1-b314-3008b9c2aad8\") " pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:41.516553 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:41.516504 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b518c36-05fd-48af-a2a3-1571be5cb852-webhook-cert\") pod \"maas-controller-5bf9578fbb-5grln\" (UID: \"9b518c36-05fd-48af-a2a3-1571be5cb852\") " pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:24:41.518901 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:41.518879 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b518c36-05fd-48af-a2a3-1571be5cb852-webhook-cert\") pod \"maas-controller-5bf9578fbb-5grln\" (UID: \"9b518c36-05fd-48af-a2a3-1571be5cb852\") " pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:24:41.607606 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:41.607568 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:41.738854 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:41.738824 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:24:41.754840 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:41.754812 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dbc579747-drs6s"] Jul 07 11:24:41.756084 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:24:41.756062 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda889086f_4bbf_41a1_b314_3008b9c2aad8.slice/crio-fc07c8019187489e5cb0384a9787955fd2805c5123f201c7ba54d1124ad92ddb WatchSource:0}: Error finding container fc07c8019187489e5cb0384a9787955fd2805c5123f201c7ba54d1124ad92ddb: Status 404 returned error can't find the container with id fc07c8019187489e5cb0384a9787955fd2805c5123f201c7ba54d1124ad92ddb Jul 07 11:24:42.096802 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:42.096776 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bf9578fbb-5grln"] Jul 07 11:24:42.098020 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:24:42.097990 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b518c36_05fd_48af_a2a3_1571be5cb852.slice/crio-f52352013fe1cad1aeaf313ed7c29d11f1244e27cdc01fbd78b0f90a52bff8dc WatchSource:0}: Error finding container f52352013fe1cad1aeaf313ed7c29d11f1244e27cdc01fbd78b0f90a52bff8dc: Status 404 returned error can't find the container with id f52352013fe1cad1aeaf313ed7c29d11f1244e27cdc01fbd78b0f90a52bff8dc Jul 07 11:24:42.379953 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:42.379918 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf9578fbb-5grln" event={"ID":"9b518c36-05fd-48af-a2a3-1571be5cb852","Type":"ContainerStarted","Data":"f52352013fe1cad1aeaf313ed7c29d11f1244e27cdc01fbd78b0f90a52bff8dc"} Jul 07 11:24:42.381428 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:42.381399 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dbc579747-drs6s" event={"ID":"a889086f-4bbf-41a1-b314-3008b9c2aad8","Type":"ContainerStarted","Data":"fc07c8019187489e5cb0384a9787955fd2805c5123f201c7ba54d1124ad92ddb"} Jul 07 11:24:44.046674 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:44.046651 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 11:24:44.397272 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:44.397236 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dbc579747-drs6s" event={"ID":"a889086f-4bbf-41a1-b314-3008b9c2aad8","Type":"ContainerStarted","Data":"6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1"} Jul 07 11:24:44.397475 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:44.397397 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:44.398815 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:44.398794 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf9578fbb-5grln" event={"ID":"9b518c36-05fd-48af-a2a3-1571be5cb852","Type":"ContainerStarted","Data":"0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52"} Jul 07 11:24:44.398929 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:44.398833 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:24:44.415827 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:44.415778 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5dbc579747-drs6s" podStartSLOduration=2.121480694 podStartE2EDuration="4.415760807s" totalImagesPullingTime="2.294280113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:24:44.414651992 +0000 UTC m=+542.002518482" watchObservedRunningTime="2026-07-07 11:24:44.415760807 +0000 UTC m=+542.003627297" Jul 07 11:24:44.437807 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:44.437764 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bf9578fbb-5grln" podStartSLOduration=2.493521418 podStartE2EDuration="4.437742151s" totalImagesPullingTime="1.944220733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:24:44.435494752 +0000 UTC m=+542.023361247" watchObservedRunningTime="2026-07-07 11:24:44.437742151 +0000 UTC m=+542.025608641" Jul 07 11:24:55.414398 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:55.414365 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:24:55.414826 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:55.414438 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:55.572194 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:55.572166 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5dbc579747-drs6s"] Jul 07 11:24:55.572392 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:55.572340 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5dbc579747-drs6s" podUID="a889086f-4bbf-41a1-b314-3008b9c2aad8" containerName="manager" containerID="cri-o://6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1" gracePeriod=30 Jul 07 11:24:55.822156 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:55.822134 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:55.953787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:55.953687 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz5g6\" (UniqueName: \"kubernetes.io/projected/a889086f-4bbf-41a1-b314-3008b9c2aad8-kube-api-access-sz5g6\") pod \"a889086f-4bbf-41a1-b314-3008b9c2aad8\" (UID: \"a889086f-4bbf-41a1-b314-3008b9c2aad8\") " Jul 07 11:24:55.953965 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:55.953815 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a889086f-4bbf-41a1-b314-3008b9c2aad8-webhook-cert\") pod \"a889086f-4bbf-41a1-b314-3008b9c2aad8\" (UID: \"a889086f-4bbf-41a1-b314-3008b9c2aad8\") " Jul 07 11:24:55.955945 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:55.955912 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a889086f-4bbf-41a1-b314-3008b9c2aad8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a889086f-4bbf-41a1-b314-3008b9c2aad8" (UID: "a889086f-4bbf-41a1-b314-3008b9c2aad8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:24:55.955945 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:55.955926 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a889086f-4bbf-41a1-b314-3008b9c2aad8-kube-api-access-sz5g6" (OuterVolumeSpecName: "kube-api-access-sz5g6") pod "a889086f-4bbf-41a1-b314-3008b9c2aad8" (UID: "a889086f-4bbf-41a1-b314-3008b9c2aad8"). InnerVolumeSpecName "kube-api-access-sz5g6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:24:56.054538 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:56.054498 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz5g6\" (UniqueName: \"kubernetes.io/projected/a889086f-4bbf-41a1-b314-3008b9c2aad8-kube-api-access-sz5g6\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:24:56.054538 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:56.054530 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a889086f-4bbf-41a1-b314-3008b9c2aad8-webhook-cert\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:24:56.464435 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:56.464395 2585 generic.go:356] "Generic (PLEG): container finished" podID="a889086f-4bbf-41a1-b314-3008b9c2aad8" containerID="6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1" exitCode=0 Jul 07 11:24:56.464934 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:56.464439 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dbc579747-drs6s" Jul 07 11:24:56.464934 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:56.464480 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dbc579747-drs6s" event={"ID":"a889086f-4bbf-41a1-b314-3008b9c2aad8","Type":"ContainerDied","Data":"6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1"} Jul 07 11:24:56.464934 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:56.464528 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dbc579747-drs6s" event={"ID":"a889086f-4bbf-41a1-b314-3008b9c2aad8","Type":"ContainerDied","Data":"fc07c8019187489e5cb0384a9787955fd2805c5123f201c7ba54d1124ad92ddb"} Jul 07 11:24:56.464934 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:56.464551 2585 scope.go:117] "RemoveContainer" containerID="6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1" Jul 07 11:24:56.473743 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:56.473726 2585 scope.go:117] "RemoveContainer" containerID="6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1" Jul 07 11:24:56.473994 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:24:56.473976 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1\": container with ID starting with 6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1 not found: ID does not exist" containerID="6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1" Jul 07 11:24:56.474066 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:56.474002 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1"} err="failed to get container status \"6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1\": rpc error: code = NotFound desc = could not find container \"6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1\": container with ID starting with 6f22fc867822b11ca322a7317e2cd9cc373f4c38b00e0fda5c9eeaae7b242ed1 not found: ID does not exist" Jul 07 11:24:56.493398 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:56.493369 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5dbc579747-drs6s"] Jul 07 11:24:56.499933 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:56.499907 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5dbc579747-drs6s"] Jul 07 11:24:57.676377 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:24:57.676325 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a889086f-4bbf-41a1-b314-3008b9c2aad8" path="/var/lib/kubelet/pods/a889086f-4bbf-41a1-b314-3008b9c2aad8/volumes" Jul 07 11:25:18.024792 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.024761 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-7f87fb4d85-ncsqs"] Jul 07 11:25:18.025311 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.025144 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a889086f-4bbf-41a1-b314-3008b9c2aad8" containerName="manager" Jul 07 11:25:18.025311 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.025158 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a889086f-4bbf-41a1-b314-3008b9c2aad8" containerName="manager" Jul 07 11:25:18.025311 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.025217 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a889086f-4bbf-41a1-b314-3008b9c2aad8" containerName="manager" Jul 07 11:25:18.031213 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.031188 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:18.034241 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.034217 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 07 11:25:18.035359 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.035328 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-fpmsd\"" Jul 07 11:25:18.051215 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.051185 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krj98\" (UniqueName: \"kubernetes.io/projected/456ad5d7-b828-4e08-aa21-244ea13bb363-kube-api-access-krj98\") pod \"maas-api-7f87fb4d85-ncsqs\" (UID: \"456ad5d7-b828-4e08-aa21-244ea13bb363\") " pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:18.051368 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.051231 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/456ad5d7-b828-4e08-aa21-244ea13bb363-maas-api-tls\") pod \"maas-api-7f87fb4d85-ncsqs\" (UID: \"456ad5d7-b828-4e08-aa21-244ea13bb363\") " pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:18.051368 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.051331 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/456ad5d7-b828-4e08-aa21-244ea13bb363-openshift-service-ca\") pod \"maas-api-7f87fb4d85-ncsqs\" (UID: \"456ad5d7-b828-4e08-aa21-244ea13bb363\") " pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:18.055037 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.055014 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7f87fb4d85-ncsqs"] Jul 07 11:25:18.152269 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.152235 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/456ad5d7-b828-4e08-aa21-244ea13bb363-maas-api-tls\") pod \"maas-api-7f87fb4d85-ncsqs\" (UID: \"456ad5d7-b828-4e08-aa21-244ea13bb363\") " pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:18.152476 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.152336 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/456ad5d7-b828-4e08-aa21-244ea13bb363-openshift-service-ca\") pod \"maas-api-7f87fb4d85-ncsqs\" (UID: \"456ad5d7-b828-4e08-aa21-244ea13bb363\") " pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:18.152476 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.152392 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krj98\" (UniqueName: \"kubernetes.io/projected/456ad5d7-b828-4e08-aa21-244ea13bb363-kube-api-access-krj98\") pod \"maas-api-7f87fb4d85-ncsqs\" (UID: \"456ad5d7-b828-4e08-aa21-244ea13bb363\") " pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:18.152975 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.152953 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/456ad5d7-b828-4e08-aa21-244ea13bb363-openshift-service-ca\") pod \"maas-api-7f87fb4d85-ncsqs\" (UID: \"456ad5d7-b828-4e08-aa21-244ea13bb363\") " pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:18.154764 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.154741 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/456ad5d7-b828-4e08-aa21-244ea13bb363-maas-api-tls\") pod \"maas-api-7f87fb4d85-ncsqs\" (UID: \"456ad5d7-b828-4e08-aa21-244ea13bb363\") " pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:18.173360 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.173330 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krj98\" (UniqueName: \"kubernetes.io/projected/456ad5d7-b828-4e08-aa21-244ea13bb363-kube-api-access-krj98\") pod \"maas-api-7f87fb4d85-ncsqs\" (UID: \"456ad5d7-b828-4e08-aa21-244ea13bb363\") " pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:18.341588 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.341499 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:18.493059 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.493033 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7f87fb4d85-ncsqs"] Jul 07 11:25:18.494916 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:25:18.494888 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod456ad5d7_b828_4e08_aa21_244ea13bb363.slice/crio-c176845f52299b33885309b7ee49b36a473a78928ffe894fab97b6fbaa9876d6 WatchSource:0}: Error finding container c176845f52299b33885309b7ee49b36a473a78928ffe894fab97b6fbaa9876d6: Status 404 returned error can't find the container with id c176845f52299b33885309b7ee49b36a473a78928ffe894fab97b6fbaa9876d6 Jul 07 11:25:18.584581 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.584547 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7f87fb4d85-ncsqs" event={"ID":"456ad5d7-b828-4e08-aa21-244ea13bb363","Type":"ContainerStarted","Data":"c176845f52299b33885309b7ee49b36a473a78928ffe894fab97b6fbaa9876d6"} Jul 07 11:25:18.935555 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.935519 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-c2vz2"] Jul 07 11:25:18.941233 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.941208 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" Jul 07 11:25:18.946508 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.946488 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zx6fz\"" Jul 07 11:25:18.947628 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.947607 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 11:25:18.958532 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.958509 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-c2vz2"] Jul 07 11:25:18.960095 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.960071 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wqh4\" (UniqueName: \"kubernetes.io/projected/e1387eae-57e8-49be-a118-c16b9895c9a6-kube-api-access-5wqh4\") pod \"payload-pre-processing-7964457696-c2vz2\" (UID: \"e1387eae-57e8-49be-a118-c16b9895c9a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" Jul 07 11:25:18.960205 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.960176 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1387eae-57e8-49be-a118-c16b9895c9a6-plugins-config-volume\") pod \"payload-pre-processing-7964457696-c2vz2\" (UID: \"e1387eae-57e8-49be-a118-c16b9895c9a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" Jul 07 11:25:18.999971 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:18.999937 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-g8ws9"] Jul 07 11:25:19.003912 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.003891 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" Jul 07 11:25:19.027892 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.027861 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-g8ws9"] Jul 07 11:25:19.060629 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.060600 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/449f9089-8fe5-4a93-8ffd-6400b4aa9e2c-plugins-config-volume\") pod \"payload-processing-58f86c7798-g8ws9\" (UID: \"449f9089-8fe5-4a93-8ffd-6400b4aa9e2c\") " pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" Jul 07 11:25:19.060802 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.060650 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1387eae-57e8-49be-a118-c16b9895c9a6-plugins-config-volume\") pod \"payload-pre-processing-7964457696-c2vz2\" (UID: \"e1387eae-57e8-49be-a118-c16b9895c9a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" Jul 07 11:25:19.060802 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.060741 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wqh4\" (UniqueName: \"kubernetes.io/projected/e1387eae-57e8-49be-a118-c16b9895c9a6-kube-api-access-5wqh4\") pod \"payload-pre-processing-7964457696-c2vz2\" (UID: \"e1387eae-57e8-49be-a118-c16b9895c9a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" Jul 07 11:25:19.060802 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.060769 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5mv6\" (UniqueName: \"kubernetes.io/projected/449f9089-8fe5-4a93-8ffd-6400b4aa9e2c-kube-api-access-d5mv6\") pod \"payload-processing-58f86c7798-g8ws9\" (UID: \"449f9089-8fe5-4a93-8ffd-6400b4aa9e2c\") " pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" Jul 07 11:25:19.061373 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.061335 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1387eae-57e8-49be-a118-c16b9895c9a6-plugins-config-volume\") pod \"payload-pre-processing-7964457696-c2vz2\" (UID: \"e1387eae-57e8-49be-a118-c16b9895c9a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" Jul 07 11:25:19.074460 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.074431 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wqh4\" (UniqueName: \"kubernetes.io/projected/e1387eae-57e8-49be-a118-c16b9895c9a6-kube-api-access-5wqh4\") pod \"payload-pre-processing-7964457696-c2vz2\" (UID: \"e1387eae-57e8-49be-a118-c16b9895c9a6\") " pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" Jul 07 11:25:19.161378 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.161325 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5mv6\" (UniqueName: \"kubernetes.io/projected/449f9089-8fe5-4a93-8ffd-6400b4aa9e2c-kube-api-access-d5mv6\") pod \"payload-processing-58f86c7798-g8ws9\" (UID: \"449f9089-8fe5-4a93-8ffd-6400b4aa9e2c\") " pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" Jul 07 11:25:19.161558 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.161430 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/449f9089-8fe5-4a93-8ffd-6400b4aa9e2c-plugins-config-volume\") pod \"payload-processing-58f86c7798-g8ws9\" (UID: \"449f9089-8fe5-4a93-8ffd-6400b4aa9e2c\") " pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" Jul 07 11:25:19.162289 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.162267 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/449f9089-8fe5-4a93-8ffd-6400b4aa9e2c-plugins-config-volume\") pod \"payload-processing-58f86c7798-g8ws9\" (UID: \"449f9089-8fe5-4a93-8ffd-6400b4aa9e2c\") " pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" Jul 07 11:25:19.176513 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.176475 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5mv6\" (UniqueName: \"kubernetes.io/projected/449f9089-8fe5-4a93-8ffd-6400b4aa9e2c-kube-api-access-d5mv6\") pod \"payload-processing-58f86c7798-g8ws9\" (UID: \"449f9089-8fe5-4a93-8ffd-6400b4aa9e2c\") " pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" Jul 07 11:25:19.251859 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.251773 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" Jul 07 11:25:19.315191 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.314872 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" Jul 07 11:25:19.483239 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.483160 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-c2vz2"] Jul 07 11:25:19.545398 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.545288 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-g8ws9"] Jul 07 11:25:19.592400 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.592358 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" event={"ID":"449f9089-8fe5-4a93-8ffd-6400b4aa9e2c","Type":"ContainerStarted","Data":"65f16017594937de85ab3bc41b432713b0d889c8456126f56acf5a8c8483f33e"} Jul 07 11:25:19.593974 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:19.593946 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" event={"ID":"e1387eae-57e8-49be-a118-c16b9895c9a6","Type":"ContainerStarted","Data":"40edf0e4cf1b70a05997bcfc6c91f6f4eeb93ae2deecd8e2b1ddf13b79cff2fa"} Jul 07 11:25:20.605724 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:20.605651 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7f87fb4d85-ncsqs" event={"ID":"456ad5d7-b828-4e08-aa21-244ea13bb363","Type":"ContainerStarted","Data":"2e1f0c0ebac4b9b1cd5f1b1a4a0439cd6a90a96076ab6ce2ba4ac6819432d99b"} Jul 07 11:25:20.606532 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:20.605857 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:20.647195 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:20.640271 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-7f87fb4d85-ncsqs" podStartSLOduration=1.917645615 podStartE2EDuration="3.640246309s" totalImagesPullingTime="1.722600694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:25:20.638746339 +0000 UTC m=+578.226612832" watchObservedRunningTime="2026-07-07 11:25:20.640246309 +0000 UTC m=+578.228112800" Jul 07 11:25:21.612541 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:21.612497 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" event={"ID":"e1387eae-57e8-49be-a118-c16b9895c9a6","Type":"ContainerStarted","Data":"8514e36f1b8f57b039e1f305a0df9ddf3877bc333f92ee88990e5bcd027518bc"} Jul 07 11:25:21.612977 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:21.612684 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" Jul 07 11:25:21.614011 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:21.613988 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" event={"ID":"449f9089-8fe5-4a93-8ffd-6400b4aa9e2c","Type":"ContainerStarted","Data":"bb3ae5b783143d23201302a7c887bf8b28f0e536debb04865754694bc06ba779"} Jul 07 11:25:21.614132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:21.614114 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" Jul 07 11:25:21.642609 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:21.642477 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" podStartSLOduration=1.7450496210000002 podStartE2EDuration="3.642454888s" totalImagesPullingTime="1.897405267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:25:21.637665019 +0000 UTC m=+579.225531511" watchObservedRunningTime="2026-07-07 11:25:21.642454888 +0000 UTC m=+579.230321379" Jul 07 11:25:21.669787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:21.669727 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" podStartSLOduration=1.841227903 podStartE2EDuration="3.669706302s" totalImagesPullingTime="1.828478399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:25:21.668724348 +0000 UTC m=+579.256590839" watchObservedRunningTime="2026-07-07 11:25:21.669706302 +0000 UTC m=+579.257572794" Jul 07 11:25:26.619815 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:26.619789 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-7f87fb4d85-ncsqs" Jul 07 11:25:26.650099 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:26.650060 2585 generic.go:356] "Generic (PLEG): container finished" podID="d5601dd8-8e67-4426-bdbf-cf49dfb965e1" containerID="a6f0e9ce1481e46365291e2eac7751136f5849e33dd8e1994163509a63ff3763" exitCode=0 Jul 07 11:25:26.650266 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:26.650137 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" event={"ID":"d5601dd8-8e67-4426-bdbf-cf49dfb965e1","Type":"ContainerDied","Data":"a6f0e9ce1481e46365291e2eac7751136f5849e33dd8e1994163509a63ff3763"} Jul 07 11:25:26.650616 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:26.650598 2585 scope.go:117] "RemoveContainer" containerID="a6f0e9ce1481e46365291e2eac7751136f5849e33dd8e1994163509a63ff3763" Jul 07 11:25:26.651814 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:26.651785 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 11:25:26.651908 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:26.651853 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 11:25:26.651908 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:26.651886 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 11:25:27.661131 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:27.661087 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" event={"ID":"d5601dd8-8e67-4426-bdbf-cf49dfb965e1","Type":"ContainerStarted","Data":"e137c17f9c01c5bc36c18e2f812ccf87582ff748a9d0c9c62460c741929780a3"} Jul 07 11:25:27.944965 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:27.944871 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:25:27.945333 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:27.945308 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wjnfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 07 11:25:27.945549 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:27.945523 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" podUID="d5601dd8-8e67-4426-bdbf-cf49dfb965e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 07 11:25:28.944184 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:28.944141 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wjnfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 07 11:25:28.944695 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:28.944207 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" podUID="d5601dd8-8e67-4426-bdbf-cf49dfb965e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 07 11:25:29.944485 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:29.944445 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wjnfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 07 11:25:29.945095 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:29.945065 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" podUID="d5601dd8-8e67-4426-bdbf-cf49dfb965e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 07 11:25:30.944601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:30.944552 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wjnfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 07 11:25:30.945114 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:30.944625 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" podUID="d5601dd8-8e67-4426-bdbf-cf49dfb965e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 07 11:25:31.113460 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:31.113423 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:25:31.948159 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:31.948130 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:25:31.949089 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:31.949068 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wjnfd" Jul 07 11:25:32.628157 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:32.628126 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-g8ws9" Jul 07 11:25:32.628377 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:32.628334 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-c2vz2" Jul 07 11:25:43.521849 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:43.521817 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:25:43.522291 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:43.521856 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:25:47.344331 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.344301 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm"] Jul 07 11:25:47.348854 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.348836 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.356061 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.356040 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 11:25:47.356823 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.356800 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 11:25:47.356936 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.356879 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5gs4g\"" Jul 07 11:25:47.356936 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.356882 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 11:25:47.373045 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.373012 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm"] Jul 07 11:25:47.422106 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.422065 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.422106 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.422108 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.422377 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.422174 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.422377 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.422227 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psqb4\" (UniqueName: \"kubernetes.io/projected/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-kube-api-access-psqb4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.422377 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.422255 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.422377 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.422325 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.523872 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.523802 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.524132 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.524114 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.524235 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.524197 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.524406 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.524388 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.524573 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.524557 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psqb4\" (UniqueName: \"kubernetes.io/projected/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-kube-api-access-psqb4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.524684 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.524670 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.524800 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.524786 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.525245 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.525224 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.525740 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.525721 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.529212 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.528513 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.533332 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.533304 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.537896 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.537874 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psqb4\" (UniqueName: \"kubernetes.io/projected/32f68e57-0e10-46d0-ab84-ac9fedb8cb1e-kube-api-access-psqb4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm\" (UID: \"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.660775 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.660724 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:47.806539 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:47.806194 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm"] Jul 07 11:25:47.810929 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:25:47.810884 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32f68e57_0e10_46d0_ab84_ac9fedb8cb1e.slice/crio-6a19a3308ffdaa411272c839e97319c9640174e2034da52b4ea59f03f053fafd WatchSource:0}: Error finding container 6a19a3308ffdaa411272c839e97319c9640174e2034da52b4ea59f03f053fafd: Status 404 returned error can't find the container with id 6a19a3308ffdaa411272c839e97319c9640174e2034da52b4ea59f03f053fafd Jul 07 11:25:48.781318 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:48.781281 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" event={"ID":"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e","Type":"ContainerStarted","Data":"6a19a3308ffdaa411272c839e97319c9640174e2034da52b4ea59f03f053fafd"} Jul 07 11:25:49.788432 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:49.788397 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" event={"ID":"32f68e57-0e10-46d0-ab84-ac9fedb8cb1e","Type":"ContainerStarted","Data":"6e23303842285ebde037f04cebc0cb2631efa4172bcc664f86c6f6080e6418b9"} Jul 07 11:25:49.808103 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:49.808037 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" podStartSLOduration=1.593704061 podStartE2EDuration="2.808014902s" totalImagesPullingTime="1.214310841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:25:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:25:49.807737119 +0000 UTC m=+607.395603610" watchObservedRunningTime="2026-07-07 11:25:49.808014902 +0000 UTC m=+607.395881392" Jul 07 11:25:51.585364 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.585316 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc"] Jul 07 11:25:51.589795 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.589772 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.592935 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.592909 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 11:25:51.600337 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.600316 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc"] Jul 07 11:25:51.667932 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.667900 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9911ed5-4d28-48c3-8bb3-d5973571f832-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.668102 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.667946 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.668102 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.668021 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.668102 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.668045 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z8xh\" (UniqueName: \"kubernetes.io/projected/f9911ed5-4d28-48c3-8bb3-d5973571f832-kube-api-access-8z8xh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.668102 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.668080 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.668263 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.668136 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.769216 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.769184 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9911ed5-4d28-48c3-8bb3-d5973571f832-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.769407 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.769233 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.769407 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.769278 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.769407 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.769305 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z8xh\" (UniqueName: \"kubernetes.io/projected/f9911ed5-4d28-48c3-8bb3-d5973571f832-kube-api-access-8z8xh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.769559 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.769403 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.769559 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.769447 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.769821 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.769791 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.769956 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.769857 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.770007 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.769977 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.771620 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.771597 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9911ed5-4d28-48c3-8bb3-d5973571f832-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.771802 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.771784 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9911ed5-4d28-48c3-8bb3-d5973571f832-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.777771 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.777748 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z8xh\" (UniqueName: \"kubernetes.io/projected/f9911ed5-4d28-48c3-8bb3-d5973571f832-kube-api-access-8z8xh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gdwtc\" (UID: \"f9911ed5-4d28-48c3-8bb3-d5973571f832\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:51.902862 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:51.902832 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:25:52.043407 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:52.043386 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc"] Jul 07 11:25:52.045323 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:25:52.045295 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9911ed5_4d28_48c3_8bb3_d5973571f832.slice/crio-c6483c80fc7ceace3eed7cb5a82ada9f0c4b78d03c522184f31f144e4ac5b17c WatchSource:0}: Error finding container c6483c80fc7ceace3eed7cb5a82ada9f0c4b78d03c522184f31f144e4ac5b17c: Status 404 returned error can't find the container with id c6483c80fc7ceace3eed7cb5a82ada9f0c4b78d03c522184f31f144e4ac5b17c Jul 07 11:25:52.047145 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:52.047127 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:25:52.807559 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:52.807524 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" event={"ID":"f9911ed5-4d28-48c3-8bb3-d5973571f832","Type":"ContainerStarted","Data":"06896e9768f3aa667cfe9ffb2943b1e905c3a70c96fa12c6d8125c31d7abbe04"} Jul 07 11:25:52.807932 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:52.807611 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" event={"ID":"f9911ed5-4d28-48c3-8bb3-d5973571f832","Type":"ContainerStarted","Data":"c6483c80fc7ceace3eed7cb5a82ada9f0c4b78d03c522184f31f144e4ac5b17c"} Jul 07 11:25:52.830815 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:52.830748 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" podStartSLOduration=1.635376132 podStartE2EDuration="1.830717866s" totalImagesPullingTime="195.341734ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:25:52.828949328 +0000 UTC m=+610.416815821" watchObservedRunningTime="2026-07-07 11:25:52.830717866 +0000 UTC m=+610.418584358" Jul 07 11:25:57.661324 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:57.661291 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:57.661749 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:57.661591 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:57.677071 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:57.677049 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:25:57.846834 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:25:57.846806 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm" Jul 07 11:26:01.903830 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:01.903795 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:26:01.904199 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:01.903867 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:26:01.916328 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:01.916303 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:26:02.872656 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:02.872631 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gdwtc" Jul 07 11:26:36.927597 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:36.927509 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7"] Jul 07 11:26:36.931179 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:36.931156 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:36.934221 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:36.934186 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 11:26:36.945564 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:36.945538 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7"] Jul 07 11:26:37.072526 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.072482 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.072826 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.072550 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.072826 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.072582 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.072826 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.072610 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.072826 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.072708 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv2mw\" (UniqueName: \"kubernetes.io/projected/a4984a2e-a39c-423a-81ff-80f040c26e88-kube-api-access-xv2mw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.072826 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.072794 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4984a2e-a39c-423a-81ff-80f040c26e88-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.173680 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.173628 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.173680 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.173685 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.173903 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.173802 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv2mw\" (UniqueName: \"kubernetes.io/projected/a4984a2e-a39c-423a-81ff-80f040c26e88-kube-api-access-xv2mw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.173903 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.173838 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4984a2e-a39c-423a-81ff-80f040c26e88-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.173976 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.173959 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.174046 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.174026 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.174103 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.174047 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.174310 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.174289 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.174425 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.174371 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.175932 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.175914 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4984a2e-a39c-423a-81ff-80f040c26e88-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.176310 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.176291 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4984a2e-a39c-423a-81ff-80f040c26e88-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.182133 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.182079 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv2mw\" (UniqueName: \"kubernetes.io/projected/a4984a2e-a39c-423a-81ff-80f040c26e88-kube-api-access-xv2mw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7\" (UID: \"a4984a2e-a39c-423a-81ff-80f040c26e88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.242473 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.242436 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:37.387541 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:37.387460 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7"] Jul 07 11:26:37.389946 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:26:37.389919 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4984a2e_a39c_423a_81ff_80f040c26e88.slice/crio-cdf4881c58fc450bf1e87df5cc510d45b816949eec9641060c0902201aaee319 WatchSource:0}: Error finding container cdf4881c58fc450bf1e87df5cc510d45b816949eec9641060c0902201aaee319: Status 404 returned error can't find the container with id cdf4881c58fc450bf1e87df5cc510d45b816949eec9641060c0902201aaee319 Jul 07 11:26:38.054112 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:38.054010 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" event={"ID":"a4984a2e-a39c-423a-81ff-80f040c26e88","Type":"ContainerStarted","Data":"76bf4f4169d367fd6c71ea2f9b8278205da219b8c00f6c28a28c2b0c6ffca8d6"} Jul 07 11:26:38.054112 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:38.054108 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" event={"ID":"a4984a2e-a39c-423a-81ff-80f040c26e88","Type":"ContainerStarted","Data":"cdf4881c58fc450bf1e87df5cc510d45b816949eec9641060c0902201aaee319"} Jul 07 11:26:38.076153 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:38.076105 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" podStartSLOduration=1.7447441019999999 podStartE2EDuration="2.076090595s" totalImagesPullingTime="331.346493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:26:38.074890435 +0000 UTC m=+655.662756927" watchObservedRunningTime="2026-07-07 11:26:38.076090595 +0000 UTC m=+655.663957085" Jul 07 11:26:47.243447 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:47.243412 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:47.243819 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:47.243474 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:47.256500 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:47.256467 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:26:48.121095 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:26:48.121061 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7" Jul 07 11:27:09.571087 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.571044 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc"] Jul 07 11:27:09.575115 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.575092 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.578203 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.578170 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 07 11:27:09.585944 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.585911 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc"] Jul 07 11:27:09.676288 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.676250 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6rgd\" (UniqueName: \"kubernetes.io/projected/c0197b3e-cd3b-4021-823f-ebac1f371a1b-kube-api-access-v6rgd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.676475 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.676295 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.676475 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.676366 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.676475 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.676455 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0197b3e-cd3b-4021-823f-ebac1f371a1b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.676603 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.676492 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.676603 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.676511 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.777452 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.777412 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.777452 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.777454 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.777767 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.777529 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6rgd\" (UniqueName: \"kubernetes.io/projected/c0197b3e-cd3b-4021-823f-ebac1f371a1b-kube-api-access-v6rgd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.777767 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.777578 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.777767 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.777609 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.777767 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.777711 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0197b3e-cd3b-4021-823f-ebac1f371a1b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.777986 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.777865 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.777986 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.777948 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.778058 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.778007 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.779902 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.779875 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0197b3e-cd3b-4021-823f-ebac1f371a1b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.780222 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.780202 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0197b3e-cd3b-4021-823f-ebac1f371a1b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.786844 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.786820 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6rgd\" (UniqueName: \"kubernetes.io/projected/c0197b3e-cd3b-4021-823f-ebac1f371a1b-kube-api-access-v6rgd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc\" (UID: \"c0197b3e-cd3b-4021-823f-ebac1f371a1b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:09.886657 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:09.886625 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:10.031748 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:10.029756 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc"] Jul 07 11:27:10.035286 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:27:10.035253 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0197b3e_cd3b_4021_823f_ebac1f371a1b.slice/crio-0f5ec2d3f1af6687d4e01ffa5e392d2591c83be493c58479fb65de95ed5f739d WatchSource:0}: Error finding container 0f5ec2d3f1af6687d4e01ffa5e392d2591c83be493c58479fb65de95ed5f739d: Status 404 returned error can't find the container with id 0f5ec2d3f1af6687d4e01ffa5e392d2591c83be493c58479fb65de95ed5f739d Jul 07 11:27:10.237115 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:10.237021 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" event={"ID":"c0197b3e-cd3b-4021-823f-ebac1f371a1b","Type":"ContainerStarted","Data":"0f5ec2d3f1af6687d4e01ffa5e392d2591c83be493c58479fb65de95ed5f739d"} Jul 07 11:27:12.251614 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:12.251580 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" event={"ID":"c0197b3e-cd3b-4021-823f-ebac1f371a1b","Type":"ContainerStarted","Data":"79c14e63cd76b414ef915526ea2a18d24b28e489bbac289d3c960fd98510466f"} Jul 07 11:27:12.273966 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:12.273901 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" podStartSLOduration=1.161026981 podStartE2EDuration="3.273883498s" totalImagesPullingTime="2.112856517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:12.272909778 +0000 UTC m=+689.860776273" watchObservedRunningTime="2026-07-07 11:27:12.273883498 +0000 UTC m=+689.861749987" Jul 07 11:27:19.887463 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:19.887430 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:19.887937 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:19.887574 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:19.900179 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:19.900153 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:27:20.308016 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:27:20.307934 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc" Jul 07 11:28:06.743745 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:06.743704 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7f69f8f5d6-kbf47"] Jul 07 11:28:06.747657 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:06.747637 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f69f8f5d6-kbf47" Jul 07 11:28:06.757285 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:06.757259 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f69f8f5d6-kbf47"] Jul 07 11:28:06.834609 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:06.834577 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5wsd\" (UniqueName: \"kubernetes.io/projected/f3cd8be0-0cff-4253-a5b2-ab9f288be269-kube-api-access-p5wsd\") pod \"authorino-7f69f8f5d6-kbf47\" (UID: \"f3cd8be0-0cff-4253-a5b2-ab9f288be269\") " pod="kuadrant-system/authorino-7f69f8f5d6-kbf47" Jul 07 11:28:06.834821 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:06.834713 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f3cd8be0-0cff-4253-a5b2-ab9f288be269-tls-cert\") pod \"authorino-7f69f8f5d6-kbf47\" (UID: \"f3cd8be0-0cff-4253-a5b2-ab9f288be269\") " pod="kuadrant-system/authorino-7f69f8f5d6-kbf47" Jul 07 11:28:06.935792 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:06.935756 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5wsd\" (UniqueName: \"kubernetes.io/projected/f3cd8be0-0cff-4253-a5b2-ab9f288be269-kube-api-access-p5wsd\") pod \"authorino-7f69f8f5d6-kbf47\" (UID: \"f3cd8be0-0cff-4253-a5b2-ab9f288be269\") " pod="kuadrant-system/authorino-7f69f8f5d6-kbf47" Jul 07 11:28:06.935967 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:06.935819 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f3cd8be0-0cff-4253-a5b2-ab9f288be269-tls-cert\") pod \"authorino-7f69f8f5d6-kbf47\" (UID: \"f3cd8be0-0cff-4253-a5b2-ab9f288be269\") " pod="kuadrant-system/authorino-7f69f8f5d6-kbf47" Jul 07 11:28:06.938177 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:06.938152 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f3cd8be0-0cff-4253-a5b2-ab9f288be269-tls-cert\") pod \"authorino-7f69f8f5d6-kbf47\" (UID: \"f3cd8be0-0cff-4253-a5b2-ab9f288be269\") " pod="kuadrant-system/authorino-7f69f8f5d6-kbf47" Jul 07 11:28:06.943726 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:06.943701 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5wsd\" (UniqueName: \"kubernetes.io/projected/f3cd8be0-0cff-4253-a5b2-ab9f288be269-kube-api-access-p5wsd\") pod \"authorino-7f69f8f5d6-kbf47\" (UID: \"f3cd8be0-0cff-4253-a5b2-ab9f288be269\") " pod="kuadrant-system/authorino-7f69f8f5d6-kbf47" Jul 07 11:28:07.057644 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:07.057558 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f69f8f5d6-kbf47" Jul 07 11:28:07.226439 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:07.226409 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f69f8f5d6-kbf47"] Jul 07 11:28:07.227833 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:28:07.227807 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3cd8be0_0cff_4253_a5b2_ab9f288be269.slice/crio-52bbbc10b4a39e20adb4703b8c5518fa96c231a238dd31b0467e62d9ff5346df WatchSource:0}: Error finding container 52bbbc10b4a39e20adb4703b8c5518fa96c231a238dd31b0467e62d9ff5346df: Status 404 returned error can't find the container with id 52bbbc10b4a39e20adb4703b8c5518fa96c231a238dd31b0467e62d9ff5346df Jul 07 11:28:07.570313 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:07.570274 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f69f8f5d6-kbf47" event={"ID":"f3cd8be0-0cff-4253-a5b2-ab9f288be269","Type":"ContainerStarted","Data":"52bbbc10b4a39e20adb4703b8c5518fa96c231a238dd31b0467e62d9ff5346df"} Jul 07 11:28:08.580804 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:08.580764 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f69f8f5d6-kbf47" event={"ID":"f3cd8be0-0cff-4253-a5b2-ab9f288be269","Type":"ContainerStarted","Data":"d9bb2de0e7429738943f669c4e6ac68face38e75855df55192ea11e60ae87496"} Jul 07 11:28:08.602632 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:08.602563 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7f69f8f5d6-kbf47" podStartSLOduration=2.15479184 podStartE2EDuration="2.602540421s" totalImagesPullingTime="447.748581ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:28:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:28:08.600173613 +0000 UTC m=+746.188040105" watchObservedRunningTime="2026-07-07 11:28:08.602540421 +0000 UTC m=+746.190406911" Jul 07 11:28:08.646372 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:08.645929 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-789c94966c-99rtj"] Jul 07 11:28:08.646372 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:08.646174 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-789c94966c-99rtj" podUID="68370995-bc5e-45a2-8dc4-84faa4bd6e2f" containerName="authorino" containerID="cri-o://0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc" gracePeriod=30 Jul 07 11:28:08.916107 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:08.916075 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-789c94966c-99rtj" Jul 07 11:28:09.056984 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.056948 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tskv\" (UniqueName: \"kubernetes.io/projected/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-kube-api-access-7tskv\") pod \"68370995-bc5e-45a2-8dc4-84faa4bd6e2f\" (UID: \"68370995-bc5e-45a2-8dc4-84faa4bd6e2f\") " Jul 07 11:28:09.057194 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.057006 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-tls-cert\") pod \"68370995-bc5e-45a2-8dc4-84faa4bd6e2f\" (UID: \"68370995-bc5e-45a2-8dc4-84faa4bd6e2f\") " Jul 07 11:28:09.059952 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.059903 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-kube-api-access-7tskv" (OuterVolumeSpecName: "kube-api-access-7tskv") pod "68370995-bc5e-45a2-8dc4-84faa4bd6e2f" (UID: "68370995-bc5e-45a2-8dc4-84faa4bd6e2f"). InnerVolumeSpecName "kube-api-access-7tskv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:28:09.077753 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.077723 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "68370995-bc5e-45a2-8dc4-84faa4bd6e2f" (UID: "68370995-bc5e-45a2-8dc4-84faa4bd6e2f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:28:09.158170 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.158071 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tskv\" (UniqueName: \"kubernetes.io/projected/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-kube-api-access-7tskv\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:28:09.158170 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.158110 2585 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68370995-bc5e-45a2-8dc4-84faa4bd6e2f-tls-cert\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:28:09.589094 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.588991 2585 generic.go:356] "Generic (PLEG): container finished" podID="68370995-bc5e-45a2-8dc4-84faa4bd6e2f" containerID="0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc" exitCode=0 Jul 07 11:28:09.589094 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.589055 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-789c94966c-99rtj" Jul 07 11:28:09.589094 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.589075 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-789c94966c-99rtj" event={"ID":"68370995-bc5e-45a2-8dc4-84faa4bd6e2f","Type":"ContainerDied","Data":"0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc"} Jul 07 11:28:09.589685 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.589117 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-789c94966c-99rtj" event={"ID":"68370995-bc5e-45a2-8dc4-84faa4bd6e2f","Type":"ContainerDied","Data":"fb6a0900cddf5f012c689265baad596e10c3ba484b2e6af8cc9ca78cab6db72f"} Jul 07 11:28:09.589685 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.589134 2585 scope.go:117] "RemoveContainer" containerID="0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc" Jul 07 11:28:09.600575 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.600554 2585 scope.go:117] "RemoveContainer" containerID="0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc" Jul 07 11:28:09.600946 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:28:09.600926 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc\": container with ID starting with 0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc not found: ID does not exist" containerID="0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc" Jul 07 11:28:09.601034 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.600959 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc"} err="failed to get container status \"0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc\": rpc error: code = NotFound desc = could not find container \"0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc\": container with ID starting with 0dbad4c41839d6b1aa8e7df322375a4d7e062e9133a865514a9059d0a7b0e7bc not found: ID does not exist" Jul 07 11:28:09.611912 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.611878 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-789c94966c-99rtj"] Jul 07 11:28:09.617479 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.617457 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-789c94966c-99rtj"] Jul 07 11:28:09.679595 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:28:09.679555 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68370995-bc5e-45a2-8dc4-84faa4bd6e2f" path="/var/lib/kubelet/pods/68370995-bc5e-45a2-8dc4-84faa4bd6e2f/volumes" Jul 07 11:29:20.069691 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.069605 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bf9578fbb-5grln"] Jul 07 11:29:20.070291 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.069893 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5bf9578fbb-5grln" podUID="9b518c36-05fd-48af-a2a3-1571be5cb852" containerName="manager" containerID="cri-o://0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52" gracePeriod=30 Jul 07 11:29:20.321688 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.321632 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:29:20.446296 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.446263 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcs9s\" (UniqueName: \"kubernetes.io/projected/9b518c36-05fd-48af-a2a3-1571be5cb852-kube-api-access-bcs9s\") pod \"9b518c36-05fd-48af-a2a3-1571be5cb852\" (UID: \"9b518c36-05fd-48af-a2a3-1571be5cb852\") " Jul 07 11:29:20.446509 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.446387 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b518c36-05fd-48af-a2a3-1571be5cb852-webhook-cert\") pod \"9b518c36-05fd-48af-a2a3-1571be5cb852\" (UID: \"9b518c36-05fd-48af-a2a3-1571be5cb852\") " Jul 07 11:29:20.448607 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.448575 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b518c36-05fd-48af-a2a3-1571be5cb852-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9b518c36-05fd-48af-a2a3-1571be5cb852" (UID: "9b518c36-05fd-48af-a2a3-1571be5cb852"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:29:20.448723 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.448650 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b518c36-05fd-48af-a2a3-1571be5cb852-kube-api-access-bcs9s" (OuterVolumeSpecName: "kube-api-access-bcs9s") pod "9b518c36-05fd-48af-a2a3-1571be5cb852" (UID: "9b518c36-05fd-48af-a2a3-1571be5cb852"). InnerVolumeSpecName "kube-api-access-bcs9s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:29:20.547824 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.547776 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcs9s\" (UniqueName: \"kubernetes.io/projected/9b518c36-05fd-48af-a2a3-1571be5cb852-kube-api-access-bcs9s\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:29:20.547824 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.547819 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b518c36-05fd-48af-a2a3-1571be5cb852-webhook-cert\") on node \"ip-10-0-135-166.ec2.internal\" DevicePath \"\"" Jul 07 11:29:20.991306 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.991273 2585 generic.go:356] "Generic (PLEG): container finished" podID="9b518c36-05fd-48af-a2a3-1571be5cb852" containerID="0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52" exitCode=0 Jul 07 11:29:20.991492 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.991319 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf9578fbb-5grln" Jul 07 11:29:20.991492 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.991384 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf9578fbb-5grln" event={"ID":"9b518c36-05fd-48af-a2a3-1571be5cb852","Type":"ContainerDied","Data":"0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52"} Jul 07 11:29:20.991492 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.991438 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf9578fbb-5grln" event={"ID":"9b518c36-05fd-48af-a2a3-1571be5cb852","Type":"ContainerDied","Data":"f52352013fe1cad1aeaf313ed7c29d11f1244e27cdc01fbd78b0f90a52bff8dc"} Jul 07 11:29:20.991492 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:20.991468 2585 scope.go:117] "RemoveContainer" containerID="0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52" Jul 07 11:29:21.001768 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.001750 2585 scope.go:117] "RemoveContainer" containerID="0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52" Jul 07 11:29:21.002014 ip-10-0-135-166 kubenswrapper[2585]: E0707 11:29:21.001997 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52\": container with ID starting with 0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52 not found: ID does not exist" containerID="0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52" Jul 07 11:29:21.002055 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.002024 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52"} err="failed to get container status \"0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52\": rpc error: code = NotFound desc = could not find container \"0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52\": container with ID starting with 0cc048c4f93ac57ed04bea46c434543c5a354c1f99fceed545c8528729b31f52 not found: ID does not exist" Jul 07 11:29:21.014441 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.014394 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bf9578fbb-5grln"] Jul 07 11:29:21.017561 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.017538 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5bf9578fbb-5grln"] Jul 07 11:29:21.675559 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.675526 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b518c36-05fd-48af-a2a3-1571be5cb852" path="/var/lib/kubelet/pods/9b518c36-05fd-48af-a2a3-1571be5cb852/volumes" Jul 07 11:29:21.751151 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.751117 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bf9578fbb-g4msv"] Jul 07 11:29:21.751760 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.751742 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b518c36-05fd-48af-a2a3-1571be5cb852" containerName="manager" Jul 07 11:29:21.751839 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.751763 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b518c36-05fd-48af-a2a3-1571be5cb852" containerName="manager" Jul 07 11:29:21.751839 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.751788 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68370995-bc5e-45a2-8dc4-84faa4bd6e2f" containerName="authorino" Jul 07 11:29:21.751839 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.751798 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="68370995-bc5e-45a2-8dc4-84faa4bd6e2f" containerName="authorino" Jul 07 11:29:21.751938 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.751907 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="68370995-bc5e-45a2-8dc4-84faa4bd6e2f" containerName="authorino" Jul 07 11:29:21.751938 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.751918 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b518c36-05fd-48af-a2a3-1571be5cb852" containerName="manager" Jul 07 11:29:21.756610 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.756594 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf9578fbb-g4msv" Jul 07 11:29:21.760681 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.760662 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 11:29:21.760803 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.760658 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dxl7v\"" Jul 07 11:29:21.760803 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.760710 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 11:29:21.764952 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.764930 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bf9578fbb-g4msv"] Jul 07 11:29:21.861884 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.861849 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73bcb144-7881-4f1f-8f78-81220d111f52-webhook-cert\") pod \"maas-controller-5bf9578fbb-g4msv\" (UID: \"73bcb144-7881-4f1f-8f78-81220d111f52\") " pod="opendatahub/maas-controller-5bf9578fbb-g4msv" Jul 07 11:29:21.862057 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.861916 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmfr6\" (UniqueName: \"kubernetes.io/projected/73bcb144-7881-4f1f-8f78-81220d111f52-kube-api-access-bmfr6\") pod \"maas-controller-5bf9578fbb-g4msv\" (UID: \"73bcb144-7881-4f1f-8f78-81220d111f52\") " pod="opendatahub/maas-controller-5bf9578fbb-g4msv" Jul 07 11:29:21.963197 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.963102 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73bcb144-7881-4f1f-8f78-81220d111f52-webhook-cert\") pod \"maas-controller-5bf9578fbb-g4msv\" (UID: \"73bcb144-7881-4f1f-8f78-81220d111f52\") " pod="opendatahub/maas-controller-5bf9578fbb-g4msv" Jul 07 11:29:21.963197 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.963191 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmfr6\" (UniqueName: \"kubernetes.io/projected/73bcb144-7881-4f1f-8f78-81220d111f52-kube-api-access-bmfr6\") pod \"maas-controller-5bf9578fbb-g4msv\" (UID: \"73bcb144-7881-4f1f-8f78-81220d111f52\") " pod="opendatahub/maas-controller-5bf9578fbb-g4msv" Jul 07 11:29:21.965510 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.965489 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73bcb144-7881-4f1f-8f78-81220d111f52-webhook-cert\") pod \"maas-controller-5bf9578fbb-g4msv\" (UID: \"73bcb144-7881-4f1f-8f78-81220d111f52\") " pod="opendatahub/maas-controller-5bf9578fbb-g4msv" Jul 07 11:29:21.972095 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:21.972067 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmfr6\" (UniqueName: \"kubernetes.io/projected/73bcb144-7881-4f1f-8f78-81220d111f52-kube-api-access-bmfr6\") pod \"maas-controller-5bf9578fbb-g4msv\" (UID: \"73bcb144-7881-4f1f-8f78-81220d111f52\") " pod="opendatahub/maas-controller-5bf9578fbb-g4msv" Jul 07 11:29:22.068277 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:22.068243 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bf9578fbb-g4msv" Jul 07 11:29:22.211045 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:22.211020 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bf9578fbb-g4msv"] Jul 07 11:29:22.212165 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:29:22.212143 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73bcb144_7881_4f1f_8f78_81220d111f52.slice/crio-fee04b8ead3b21abd153162e79fb3f040a4223608ea2ec63084fb4741460ab6d WatchSource:0}: Error finding container fee04b8ead3b21abd153162e79fb3f040a4223608ea2ec63084fb4741460ab6d: Status 404 returned error can't find the container with id fee04b8ead3b21abd153162e79fb3f040a4223608ea2ec63084fb4741460ab6d Jul 07 11:29:23.014787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:23.014687 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf9578fbb-g4msv" event={"ID":"73bcb144-7881-4f1f-8f78-81220d111f52","Type":"ContainerStarted","Data":"87de3eabcb6f3e1508d15fadaaea567c2a4dbd2d9c49d1a0a5538aa6ad0b28d3"} Jul 07 11:29:23.014787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:23.014781 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bf9578fbb-g4msv" Jul 07 11:29:23.015201 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:23.014806 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bf9578fbb-g4msv" event={"ID":"73bcb144-7881-4f1f-8f78-81220d111f52","Type":"ContainerStarted","Data":"fee04b8ead3b21abd153162e79fb3f040a4223608ea2ec63084fb4741460ab6d"} Jul 07 11:29:23.035843 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:23.035797 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bf9578fbb-g4msv" podStartSLOduration=1.56574246 podStartE2EDuration="2.035781786s" totalImagesPullingTime="470.039326ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:29:23.034498467 +0000 UTC m=+820.622364958" watchObservedRunningTime="2026-07-07 11:29:23.035781786 +0000 UTC m=+820.623648276" Jul 07 11:29:34.025951 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:29:34.025920 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bf9578fbb-g4msv" Jul 07 11:30:43.564222 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:30:43.564187 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:30:43.565842 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:30:43.565821 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:35:43.602502 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:35:43.602472 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:35:43.605205 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:35:43.605178 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:40:43.640245 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:40:43.640214 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:40:43.645044 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:40:43.645022 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:45:43.687827 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:45:43.687731 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:45:43.694768 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:45:43.694748 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:50:43.725733 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:50:43.725618 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:50:43.737002 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:50:43.736977 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:55:43.771079 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:55:43.770975 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:55:43.789084 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:55:43.789051 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sv4f_9609fab6-8573-4b1d-8a7b-add43d375ecc/ovn-acl-logging/0.log" Jul 07 11:57:09.762960 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:09.762921 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7f69f8f5d6-kbf47_f3cd8be0-0cff-4253-a5b2-ab9f288be269/authorino/0.log" Jul 07 11:57:14.380356 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:14.380326 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4r68v_5cf720f3-0e4c-4731-821c-087de85bacf8/manager/0.log" Jul 07 11:57:14.498983 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:14.498952 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-v5gtz_0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb/manager/0.log" Jul 07 11:57:14.619026 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:14.618990 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7f87fb4d85-ncsqs_456ad5d7-b828-4e08-aa21-244ea13bb363/maas-api/0.log" Jul 07 11:57:14.961712 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:14.961671 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bf9578fbb-g4msv_73bcb144-7881-4f1f-8f78-81220d111f52/manager/0.log" Jul 07 11:57:15.072592 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:15.072551 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ld52x_fd2c4e09-6656-4c98-a5cc-3451a3b01528/server/0.log" Jul 07 11:57:15.192065 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:15.192036 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-4vwpx_5dbcfd76-bbd9-44f0-98e2-b9bfac799f77/manager/0.log" Jul 07 11:57:15.421267 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:15.421242 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-99mxx_d9887f11-40f8-4adc-9cf4-e3df0ce77560/copy-manifests/0.log" Jul 07 11:57:15.440000 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:15.439968 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-99mxx_d9887f11-40f8-4adc-9cf4-e3df0ce77560/manager/0.log" Jul 07 11:57:15.690098 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:15.690010 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-tpxql_342a50aa-58ff-4643-add7-01939a5bb888/postgres/0.log" Jul 07 11:57:16.459275 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.459241 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg_e4750739-72ca-4709-bd60-0df6c7551919/util/0.log" Jul 07 11:57:16.466396 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.466367 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg_e4750739-72ca-4709-bd60-0df6c7551919/pull/0.log" Jul 07 11:57:16.472583 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.472564 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg_e4750739-72ca-4709-bd60-0df6c7551919/extract/0.log" Jul 07 11:57:16.582772 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.582742 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948_33d3b15d-9fbc-4bb6-80c7-fda597f25c8a/util/0.log" Jul 07 11:57:16.589433 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.589406 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948_33d3b15d-9fbc-4bb6-80c7-fda597f25c8a/pull/0.log" Jul 07 11:57:16.596853 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.596814 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948_33d3b15d-9fbc-4bb6-80c7-fda597f25c8a/extract/0.log" Jul 07 11:57:16.709015 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.708988 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92_a860b85d-fd65-46df-9a27-0284d3091631/pull/0.log" Jul 07 11:57:16.715430 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.715363 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92_a860b85d-fd65-46df-9a27-0284d3091631/extract/0.log" Jul 07 11:57:16.722004 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.721988 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92_a860b85d-fd65-46df-9a27-0284d3091631/util/0.log" Jul 07 11:57:16.840274 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.840247 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r_c7f054cf-9787-4d84-bcd0-c6e32623e0eb/util/0.log" Jul 07 11:57:16.847417 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.847382 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r_c7f054cf-9787-4d84-bcd0-c6e32623e0eb/pull/0.log" Jul 07 11:57:16.854874 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.854850 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r_c7f054cf-9787-4d84-bcd0-c6e32623e0eb/extract/0.log" Jul 07 11:57:16.976768 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:16.976667 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7f69f8f5d6-kbf47_f3cd8be0-0cff-4253-a5b2-ab9f288be269/authorino/0.log" Jul 07 11:57:17.144707 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:17.144676 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lsb6g_13b528bb-09b5-4927-8bc4-a4e592f25e91/manager/0.log" Jul 07 11:57:17.523292 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:17.523261 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pzqg4_101c4a72-98a9-4b15-86c3-6e8868f92247/registry-server/0.log" Jul 07 11:57:17.873698 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:17.873666 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jjclb_c18cde74-7cd5-4bac-9fd5-95ad639e84e7/manager/0.log" Jul 07 11:57:18.571418 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:18.571390 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785759898c-qqwxb_05c36606-981a-422a-8c40-4cb33f3e547f/kube-auth-proxy/0.log" Jul 07 11:57:18.691046 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:18.691016 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wjnfd_d5601dd8-8e67-4426-bdbf-cf49dfb965e1/istio-proxy/1.log" Jul 07 11:57:18.814510 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:18.814474 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-c2vz2_e1387eae-57e8-49be-a118-c16b9895c9a6/payload-pre-processing/0.log" Jul 07 11:57:18.938064 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:18.938031 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-g8ws9_449f9089-8fe5-4a93-8ffd-6400b4aa9e2c/payload-processing/0.log" Jul 07 11:57:19.379231 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:19.379193 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-4zfwm_32f68e57-0e10-46d0-ab84-ac9fedb8cb1e/main/0.log" Jul 07 11:57:19.493936 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:19.493895 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-gdwtc_f9911ed5-4d28-48c3-8bb3-d5973571f832/main/0.log" Jul 07 11:57:19.714693 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:19.714611 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7ltt7_a4984a2e-a39c-423a-81ff-80f040c26e88/main/0.log" Jul 07 11:57:19.951096 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:19.951045 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-c5bxc_c0197b3e-cd3b-4021-823f-ebac1f371a1b/main/0.log" Jul 07 11:57:24.177308 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.177275 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k5px7/must-gather-z28fb"] Jul 07 11:57:24.181014 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.180994 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5px7/must-gather-z28fb" Jul 07 11:57:24.184069 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.184049 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k5px7\"/\"default-dockercfg-7s2dd\"" Jul 07 11:57:24.184187 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.184089 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5px7\"/\"kube-root-ca.crt\"" Jul 07 11:57:24.185288 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.185270 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5px7\"/\"openshift-service-ca.crt\"" Jul 07 11:57:24.192062 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.192043 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5px7/must-gather-z28fb"] Jul 07 11:57:24.338387 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.338361 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7phw\" (UniqueName: \"kubernetes.io/projected/148a1851-2d7a-476b-93f6-dfb856ee94d2-kube-api-access-z7phw\") pod \"must-gather-z28fb\" (UID: \"148a1851-2d7a-476b-93f6-dfb856ee94d2\") " pod="openshift-must-gather-k5px7/must-gather-z28fb" Jul 07 11:57:24.338540 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.338403 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/148a1851-2d7a-476b-93f6-dfb856ee94d2-must-gather-output\") pod \"must-gather-z28fb\" (UID: \"148a1851-2d7a-476b-93f6-dfb856ee94d2\") " pod="openshift-must-gather-k5px7/must-gather-z28fb" Jul 07 11:57:24.439583 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.439492 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7phw\" (UniqueName: \"kubernetes.io/projected/148a1851-2d7a-476b-93f6-dfb856ee94d2-kube-api-access-z7phw\") pod \"must-gather-z28fb\" (UID: \"148a1851-2d7a-476b-93f6-dfb856ee94d2\") " pod="openshift-must-gather-k5px7/must-gather-z28fb" Jul 07 11:57:24.439583 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.439537 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/148a1851-2d7a-476b-93f6-dfb856ee94d2-must-gather-output\") pod \"must-gather-z28fb\" (UID: \"148a1851-2d7a-476b-93f6-dfb856ee94d2\") " pod="openshift-must-gather-k5px7/must-gather-z28fb" Jul 07 11:57:24.439824 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.439809 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/148a1851-2d7a-476b-93f6-dfb856ee94d2-must-gather-output\") pod \"must-gather-z28fb\" (UID: \"148a1851-2d7a-476b-93f6-dfb856ee94d2\") " pod="openshift-must-gather-k5px7/must-gather-z28fb" Jul 07 11:57:24.447998 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.447968 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7phw\" (UniqueName: \"kubernetes.io/projected/148a1851-2d7a-476b-93f6-dfb856ee94d2-kube-api-access-z7phw\") pod \"must-gather-z28fb\" (UID: \"148a1851-2d7a-476b-93f6-dfb856ee94d2\") " pod="openshift-must-gather-k5px7/must-gather-z28fb" Jul 07 11:57:24.491112 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.491079 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5px7/must-gather-z28fb" Jul 07 11:57:24.631902 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.631870 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5px7/must-gather-z28fb"] Jul 07 11:57:24.632873 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:57:24.632848 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod148a1851_2d7a_476b_93f6_dfb856ee94d2.slice/crio-f36850620ce68e4cece929a4a6d5001644b35a883ab485714d60d021299a2b75 WatchSource:0}: Error finding container f36850620ce68e4cece929a4a6d5001644b35a883ab485714d60d021299a2b75: Status 404 returned error can't find the container with id f36850620ce68e4cece929a4a6d5001644b35a883ab485714d60d021299a2b75 Jul 07 11:57:24.634738 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.634720 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:57:24.646295 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:24.646265 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5px7/must-gather-z28fb" event={"ID":"148a1851-2d7a-476b-93f6-dfb856ee94d2","Type":"ContainerStarted","Data":"f36850620ce68e4cece929a4a6d5001644b35a883ab485714d60d021299a2b75"} Jul 07 11:57:25.657699 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:25.657655 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5px7/must-gather-z28fb" event={"ID":"148a1851-2d7a-476b-93f6-dfb856ee94d2","Type":"ContainerStarted","Data":"7835f1116d9b5f596d2273b0a5fc3001c8ff6960c2d50e782722d248966d4201"} Jul 07 11:57:25.658164 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:25.657715 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5px7/must-gather-z28fb" event={"ID":"148a1851-2d7a-476b-93f6-dfb856ee94d2","Type":"ContainerStarted","Data":"19b8f0e07a6c44cade802012fcc76c6c5b34ab3bb91654c02f842ceac106aa01"} Jul 07 11:57:25.676456 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:25.676331 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k5px7/must-gather-z28fb" podStartSLOduration=0.891863683 podStartE2EDuration="1.676314101s" totalImagesPullingTime="784.450418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:57:25.674731012 +0000 UTC m=+2503.262597504" watchObservedRunningTime="2026-07-07 11:57:25.676314101 +0000 UTC m=+2503.264180591" Jul 07 11:57:26.965414 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:26.965379 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5zc94_45bf26b3-aacb-4588-bf31-04a417518c64/global-pull-secret-syncer/0.log" Jul 07 11:57:27.202699 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:27.202662 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qw6cr_c29cf0f1-4400-493b-8746-91f03ba47db6/konnectivity-agent/0.log" Jul 07 11:57:27.268275 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:27.268200 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-166.ec2.internal_b14f96639c7f7d472ab32a017c1adf2c/haproxy/0.log" Jul 07 11:57:31.714838 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:31.714807 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg_e4750739-72ca-4709-bd60-0df6c7551919/extract/0.log" Jul 07 11:57:31.748252 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:31.748221 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg_e4750739-72ca-4709-bd60-0df6c7551919/util/0.log" Jul 07 11:57:31.784484 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:31.784453 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lv2tg_e4750739-72ca-4709-bd60-0df6c7551919/pull/0.log" Jul 07 11:57:31.817133 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:31.817005 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948_33d3b15d-9fbc-4bb6-80c7-fda597f25c8a/extract/0.log" Jul 07 11:57:31.841627 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:31.841575 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948_33d3b15d-9fbc-4bb6-80c7-fda597f25c8a/util/0.log" Jul 07 11:57:31.873783 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:31.873705 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ss948_33d3b15d-9fbc-4bb6-80c7-fda597f25c8a/pull/0.log" Jul 07 11:57:31.926957 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:31.926929 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92_a860b85d-fd65-46df-9a27-0284d3091631/extract/0.log" Jul 07 11:57:31.989607 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:31.989514 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92_a860b85d-fd65-46df-9a27-0284d3091631/util/0.log" Jul 07 11:57:32.026091 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:32.026055 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r8n92_a860b85d-fd65-46df-9a27-0284d3091631/pull/0.log" Jul 07 11:57:32.062452 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:32.062421 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r_c7f054cf-9787-4d84-bcd0-c6e32623e0eb/extract/0.log" Jul 07 11:57:32.093284 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:32.093260 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r_c7f054cf-9787-4d84-bcd0-c6e32623e0eb/util/0.log" Jul 07 11:57:32.117431 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:32.117393 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4f9r_c7f054cf-9787-4d84-bcd0-c6e32623e0eb/pull/0.log" Jul 07 11:57:32.632423 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:32.632388 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7f69f8f5d6-kbf47_f3cd8be0-0cff-4253-a5b2-ab9f288be269/authorino/0.log" Jul 07 11:57:32.680366 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:32.678303 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lsb6g_13b528bb-09b5-4927-8bc4-a4e592f25e91/manager/0.log" Jul 07 11:57:32.801994 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:32.801953 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pzqg4_101c4a72-98a9-4b15-86c3-6e8868f92247/registry-server/0.log" Jul 07 11:57:33.099844 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:33.099756 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jjclb_c18cde74-7cd5-4bac-9fd5-95ad639e84e7/manager/0.log" Jul 07 11:57:34.710298 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:34.710265 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_76530572-a851-43b8-a567-d586360a5e8f/alertmanager/0.log" Jul 07 11:57:34.738685 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:34.738655 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_76530572-a851-43b8-a567-d586360a5e8f/config-reloader/0.log" Jul 07 11:57:34.764740 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:34.764702 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_76530572-a851-43b8-a567-d586360a5e8f/kube-rbac-proxy-web/0.log" Jul 07 11:57:34.789622 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:34.789580 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_76530572-a851-43b8-a567-d586360a5e8f/kube-rbac-proxy/0.log" Jul 07 11:57:34.815177 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:34.815137 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_76530572-a851-43b8-a567-d586360a5e8f/kube-rbac-proxy-metric/0.log" Jul 07 11:57:34.843988 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:34.843959 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_76530572-a851-43b8-a567-d586360a5e8f/prom-label-proxy/0.log" Jul 07 11:57:34.870636 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:34.870605 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_76530572-a851-43b8-a567-d586360a5e8f/init-config-reloader/0.log" Jul 07 11:57:35.045734 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:35.045647 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7ccddd76f4-g99h5_5caa2274-4ee6-4a35-839a-8b0840478ab6/metrics-server/0.log" Jul 07 11:57:35.195743 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:35.195705 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jfb75_f641be91-b0b9-498b-aba8-728d2271f9c2/node-exporter/0.log" Jul 07 11:57:35.217601 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:35.217569 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jfb75_f641be91-b0b9-498b-aba8-728d2271f9c2/kube-rbac-proxy/0.log" Jul 07 11:57:35.238014 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:35.237977 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jfb75_f641be91-b0b9-498b-aba8-728d2271f9c2/init-textfile/0.log" Jul 07 11:57:35.383296 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:35.383258 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-5cnh2_c6bab654-ad53-4cfa-8e0b-2cf60be6d410/kube-rbac-proxy-main/0.log" Jul 07 11:57:35.411323 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:35.411292 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-5cnh2_c6bab654-ad53-4cfa-8e0b-2cf60be6d410/kube-rbac-proxy-self/0.log" Jul 07 11:57:35.437090 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:35.437061 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-5cnh2_c6bab654-ad53-4cfa-8e0b-2cf60be6d410/openshift-state-metrics/0.log" Jul 07 11:57:35.703060 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:35.702979 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-9bwkw_e001e573-d466-497b-8408-9f0f7d098d57/prometheus-operator-admission-webhook/0.log" Jul 07 11:57:35.731590 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:35.731560 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f8746c6c6-8gknp_060ccbed-f9be-404c-8a7c-11e47e77646a/telemeter-client/0.log" Jul 07 11:57:35.756936 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:35.756904 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f8746c6c6-8gknp_060ccbed-f9be-404c-8a7c-11e47e77646a/reload/0.log" Jul 07 11:57:35.781311 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:35.781272 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f8746c6c6-8gknp_060ccbed-f9be-404c-8a7c-11e47e77646a/kube-rbac-proxy/0.log" Jul 07 11:57:37.406787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.406750 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr"] Jul 07 11:57:37.415380 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.415330 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.418454 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.418428 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr"] Jul 07 11:57:37.528449 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.528115 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-proc\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.528449 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.528187 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-podres\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.528449 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.528225 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnhhk\" (UniqueName: \"kubernetes.io/projected/562158aa-b99f-45c6-a104-6ceb76755c3b-kube-api-access-nnhhk\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.528449 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.528269 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-sys\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.528449 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.528300 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-lib-modules\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.629868 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.629834 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-proc\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.630787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.630071 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-proc\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.630787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.630152 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-podres\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.630787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.630195 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnhhk\" (UniqueName: \"kubernetes.io/projected/562158aa-b99f-45c6-a104-6ceb76755c3b-kube-api-access-nnhhk\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.630787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.630242 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-sys\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.630787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.630278 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-lib-modules\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.630787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.630332 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-podres\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.630787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.630462 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-sys\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.630787 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.630563 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/562158aa-b99f-45c6-a104-6ceb76755c3b-lib-modules\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.639254 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.639229 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnhhk\" (UniqueName: \"kubernetes.io/projected/562158aa-b99f-45c6-a104-6ceb76755c3b-kube-api-access-nnhhk\") pod \"perf-node-gather-daemonset-6pnvr\" (UID: \"562158aa-b99f-45c6-a104-6ceb76755c3b\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.734672 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.734571 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:37.938497 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:37.938468 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr"] Jul 07 11:57:37.941244 ip-10-0-135-166 kubenswrapper[2585]: W0707 11:57:37.941215 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod562158aa_b99f_45c6_a104_6ceb76755c3b.slice/crio-137ee4ded1ea2cfb84260b5bfbb9810198caa32a7ebfd780433d364b18083f60 WatchSource:0}: Error finding container 137ee4ded1ea2cfb84260b5bfbb9810198caa32a7ebfd780433d364b18083f60: Status 404 returned error can't find the container with id 137ee4ded1ea2cfb84260b5bfbb9810198caa32a7ebfd780433d364b18083f60 Jul 07 11:57:38.289631 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:38.289544 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-2chlh_1895fd42-75c7-4daf-8e26-07d5adae8df0/download-server/0.log" Jul 07 11:57:38.775282 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:38.775250 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" event={"ID":"562158aa-b99f-45c6-a104-6ceb76755c3b","Type":"ContainerStarted","Data":"598dd023205f4063aabffcb715e2a50d5c5220e483d90736c2b5d2d0ee0edb73"} Jul 07 11:57:38.775783 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:38.775301 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" event={"ID":"562158aa-b99f-45c6-a104-6ceb76755c3b","Type":"ContainerStarted","Data":"137ee4ded1ea2cfb84260b5bfbb9810198caa32a7ebfd780433d364b18083f60"} Jul 07 11:57:38.775783 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:38.775362 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:38.794879 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:38.794817 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" podStartSLOduration=1.79479527 podStartE2EDuration="1.79479527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:57:38.7914746 +0000 UTC m=+2516.379341091" watchObservedRunningTime="2026-07-07 11:57:38.79479527 +0000 UTC m=+2516.382661760" Jul 07 11:57:39.883068 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:39.883022 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8cpdb_7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf/dns/0.log" Jul 07 11:57:39.907903 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:39.907872 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8cpdb_7dfa4e40-8be9-4dde-91b0-9b5e812b6fbf/kube-rbac-proxy/0.log" Jul 07 11:57:40.077792 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:40.077767 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kz98j_3e2bee60-3e0b-403d-80a0-86946959866d/dns-node-resolver/0.log" Jul 07 11:57:40.623475 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:40.623442 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-bb544648c-4lkkg_3177f6fc-71ff-4a37-939a-9ffe0611172b/registry/0.log" Jul 07 11:57:40.671214 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:40.671186 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7bn6j_ef118293-20df-4a0e-8b0f-9204c6096441/node-ca/0.log" Jul 07 11:57:41.839470 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:41.839440 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785759898c-qqwxb_05c36606-981a-422a-8c40-4cb33f3e547f/kube-auth-proxy/0.log" Jul 07 11:57:41.866826 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:41.866783 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wjnfd_d5601dd8-8e67-4426-bdbf-cf49dfb965e1/istio-proxy/0.log" Jul 07 11:57:41.931793 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:41.931742 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wjnfd_d5601dd8-8e67-4426-bdbf-cf49dfb965e1/istio-proxy/1.log" Jul 07 11:57:41.974198 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:41.974170 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-c2vz2_e1387eae-57e8-49be-a118-c16b9895c9a6/payload-pre-processing/0.log" Jul 07 11:57:42.055245 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:42.055210 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-g8ws9_449f9089-8fe5-4a93-8ffd-6400b4aa9e2c/payload-processing/0.log" Jul 07 11:57:42.792187 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:42.792154 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l6xpr_44a96807-d2da-429a-9ee8-27a92649ac56/serve-healthcheck-canary/0.log" Jul 07 11:57:43.398415 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:43.398375 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8vcl_ed8c5e9b-6266-46f1-b394-53fc8ecc1056/kube-rbac-proxy/0.log" Jul 07 11:57:43.420582 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:43.420558 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8vcl_ed8c5e9b-6266-46f1-b394-53fc8ecc1056/exporter/0.log" Jul 07 11:57:43.440367 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:43.440324 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8vcl_ed8c5e9b-6266-46f1-b394-53fc8ecc1056/extractor/0.log" Jul 07 11:57:44.793726 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:44.793699 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-6pnvr" Jul 07 11:57:45.576884 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:45.576850 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4r68v_5cf720f3-0e4c-4731-821c-087de85bacf8/manager/0.log" Jul 07 11:57:45.823042 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:45.821728 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-v5gtz_0ee8a0df-9dbe-45d9-b34d-a07b259bd5fb/manager/0.log" Jul 07 11:57:45.873771 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:45.873738 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7f87fb4d85-ncsqs_456ad5d7-b828-4e08-aa21-244ea13bb363/maas-api/0.log" Jul 07 11:57:46.075257 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:46.075229 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bf9578fbb-g4msv_73bcb144-7881-4f1f-8f78-81220d111f52/manager/0.log" Jul 07 11:57:46.097902 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:46.097847 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ld52x_fd2c4e09-6656-4c98-a5cc-3451a3b01528/server/0.log" Jul 07 11:57:46.218151 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:46.218065 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-4vwpx_5dbcfd76-bbd9-44f0-98e2-b9bfac799f77/manager/0.log" Jul 07 11:57:46.383189 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:46.383163 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-99mxx_d9887f11-40f8-4adc-9cf4-e3df0ce77560/manager/0.log" Jul 07 11:57:46.402479 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:46.402429 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-99mxx_d9887f11-40f8-4adc-9cf4-e3df0ce77560/copy-manifests/0.log" Jul 07 11:57:46.478973 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:46.478887 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-tpxql_342a50aa-58ff-4643-add7-01939a5bb888/postgres/0.log" Jul 07 11:57:48.081008 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:48.080929 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-pk2q4_b23aeb58-bbe7-49da-87e8-69ef4cce1400/openshift-lws-operator/0.log" Jul 07 11:57:48.124074 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:48.124042 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785759898c-qqwxb_05c36606-981a-422a-8c40-4cb33f3e547f/kube-auth-proxy/0.log" Jul 07 11:57:48.152371 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:48.152318 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wjnfd_d5601dd8-8e67-4426-bdbf-cf49dfb965e1/istio-proxy/0.log" Jul 07 11:57:48.210592 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:48.210547 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wjnfd_d5601dd8-8e67-4426-bdbf-cf49dfb965e1/istio-proxy/1.log" Jul 07 11:57:48.246141 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:48.246107 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-c2vz2_e1387eae-57e8-49be-a118-c16b9895c9a6/payload-pre-processing/0.log" Jul 07 11:57:48.315891 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:48.315835 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-g8ws9_449f9089-8fe5-4a93-8ffd-6400b4aa9e2c/payload-processing/0.log" Jul 07 11:57:49.747821 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:49.747788 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785759898c-qqwxb_05c36606-981a-422a-8c40-4cb33f3e547f/kube-auth-proxy/0.log" Jul 07 11:57:49.782064 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:49.782034 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wjnfd_d5601dd8-8e67-4426-bdbf-cf49dfb965e1/istio-proxy/0.log" Jul 07 11:57:49.845175 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:49.845092 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wjnfd_d5601dd8-8e67-4426-bdbf-cf49dfb965e1/istio-proxy/1.log" Jul 07 11:57:49.892901 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:49.892860 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-c2vz2_e1387eae-57e8-49be-a118-c16b9895c9a6/payload-pre-processing/0.log" Jul 07 11:57:49.973738 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:49.973708 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-g8ws9_449f9089-8fe5-4a93-8ffd-6400b4aa9e2c/payload-processing/0.log" Jul 07 11:57:54.383552 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:54.383521 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9dcwb_a648f49b-76f9-44b9-9165-d96f40f7f836/kube-multus/0.log" Jul 07 11:57:54.558797 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:54.558772 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfs6d_0c92ffd4-b8cb-4485-827e-206cf7e95421/kube-multus-additional-cni-plugins/0.log" Jul 07 11:57:54.578270 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:54.578244 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfs6d_0c92ffd4-b8cb-4485-827e-206cf7e95421/egress-router-binary-copy/0.log" Jul 07 11:57:54.598242 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:54.598213 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfs6d_0c92ffd4-b8cb-4485-827e-206cf7e95421/cni-plugins/0.log" Jul 07 11:57:54.621567 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:54.621542 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfs6d_0c92ffd4-b8cb-4485-827e-206cf7e95421/bond-cni-plugin/0.log" Jul 07 11:57:54.641009 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:54.640936 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfs6d_0c92ffd4-b8cb-4485-827e-206cf7e95421/routeoverride-cni/0.log" Jul 07 11:57:54.661774 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:54.661744 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfs6d_0c92ffd4-b8cb-4485-827e-206cf7e95421/whereabouts-cni-bincopy/0.log" Jul 07 11:57:54.681218 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:54.681195 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rfs6d_0c92ffd4-b8cb-4485-827e-206cf7e95421/whereabouts-cni/0.log" Jul 07 11:57:54.901077 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:54.900959 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9rwbg_5da61a2d-4113-4869-99bf-f0d24e5edf97/network-metrics-daemon/0.log" Jul 07 11:57:54.921242 ip-10-0-135-166 kubenswrapper[2585]: I0707 11:57:54.921209 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9rwbg_5da61a2d-4113-4869-99bf-f0d24e5edf97/kube-rbac-proxy/0.log"