Jul 07 12:40:22.526050 ip-10-0-135-212 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 12:40:22.526061 ip-10-0-135-212 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 12:40:22.526068 ip-10-0-135-212 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 12:40:22.526276 ip-10-0-135-212 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 12:40:32.632860 ip-10-0-135-212 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 12:40:32.632881 ip-10-0-135-212 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e7112544cbcf41f0b52ffe24067c21d6 -- Jul 07 12:42:31.048229 ip-10-0-135-212 systemd[1]: Starting Kubernetes Kubelet... Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.495538 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500734 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500747 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500751 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500755 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500758 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500762 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500765 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:32.942452 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500769 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500774 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500777 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500780 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500783 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500786 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500796 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500800 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500802 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500806 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500810 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500813 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500816 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500820 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500823 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500826 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500829 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500831 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500834 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:32.943579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500836 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500839 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500842 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500845 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500848 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500851 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500853 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500856 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500859 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500862 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500864 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500867 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500870 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500872 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500875 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500878 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500881 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500885 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500887 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500890 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:32.944403 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500892 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500895 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500897 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500900 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500903 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500905 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500907 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500912 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500914 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500917 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500920 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500923 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500925 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500928 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500930 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500933 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500935 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500938 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500941 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500946 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:32.945235 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500961 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500965 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500967 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500970 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500973 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500976 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500979 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500982 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500985 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500989 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500992 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500995 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.500997 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501000 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501002 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501005 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501008 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501011 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501013 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501016 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:32.945856 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501019 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501021 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501024 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501026 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501029 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501031 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501034 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501036 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501039 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501041 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501044 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501046 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.501049 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503861 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503881 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503887 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503892 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503898 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503905 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:32.946473 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503912 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503917 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503921 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503926 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503930 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503935 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503939 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503965 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503970 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503975 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503979 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503984 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503988 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.503992 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504030 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504034 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504038 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504043 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504047 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504051 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:32.947095 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504059 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504064 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504068 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504072 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504076 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504080 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504086 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504090 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504094 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504098 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504102 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504107 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504116 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504122 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504126 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504130 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504135 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504139 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504146 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504151 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:32.947641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504156 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504161 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504166 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504172 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504181 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504185 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504189 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504212 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504216 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504220 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504224 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504229 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504233 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504237 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504242 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504247 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504255 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504259 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504268 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:32.948220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504273 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504278 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504282 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504287 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504291 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504296 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504300 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504305 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504309 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504317 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504321 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504326 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504330 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504334 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504338 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504343 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504347 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504351 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504355 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504359 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:32.948728 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504363 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504385 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504394 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504399 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504403 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504407 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504411 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504416 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504420 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504424 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504428 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504433 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504438 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.504442 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505195 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505204 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505207 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505210 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505213 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505216 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:32.949282 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505221 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505224 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505226 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505229 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505232 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505234 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505237 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505240 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505242 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505245 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505248 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505251 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505254 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505257 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505260 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505262 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505265 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505267 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505270 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505272 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:32.949905 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505275 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505277 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505280 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505283 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505287 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505292 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505294 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505298 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505300 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505303 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505305 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505308 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505312 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505315 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505317 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505320 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505323 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505325 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505328 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:32.950580 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505331 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505333 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505335 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505338 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505341 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505344 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505347 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505351 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505354 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505356 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505359 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505361 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505364 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505367 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505369 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505371 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505374 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505376 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505379 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:32.951561 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505381 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505385 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505388 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505390 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505392 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505395 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505399 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505402 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505405 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505409 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505413 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505416 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505418 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505421 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505424 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505426 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505429 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505431 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505434 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:32.952170 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505436 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505439 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505441 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505444 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505446 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505449 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505451 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505453 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505456 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505458 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505461 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505464 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505466 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505469 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505473 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.505475 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505545 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505552 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505558 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505564 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505569 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 12:42:33.052150 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505572 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 12:42:32.959846 ip-10-0-135-212 systemd[1]: Started Kubernetes Kubelet. Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505576 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505580 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505583 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505587 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505590 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505593 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505596 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505600 2574 flags.go:64] FLAG: --cgroup-root="" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505602 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505605 2574 flags.go:64] FLAG: --client-ca-file="" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505609 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505620 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505624 2574 flags.go:64] FLAG: --cluster-domain="" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505627 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505630 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505633 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505637 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505641 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505643 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505647 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505650 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505653 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505656 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 12:42:33.053101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505659 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505664 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505667 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505672 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505675 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505678 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505682 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505684 2574 flags.go:64] FLAG: --enable-server="true" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505688 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505692 2574 flags.go:64] FLAG: --event-burst="100" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505695 2574 flags.go:64] FLAG: --event-qps="50" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505698 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505701 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505703 2574 flags.go:64] FLAG: --eviction-hard="" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505707 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505710 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505714 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505716 2574 flags.go:64] FLAG: --eviction-soft="" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505719 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505722 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505725 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505728 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505731 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505734 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505736 2574 flags.go:64] FLAG: --feature-gates="" Jul 07 12:42:33.053985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505740 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505743 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505746 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505749 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505752 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505755 2574 flags.go:64] FLAG: --help="false" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505758 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505761 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505765 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505768 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505771 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505775 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505778 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505783 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505786 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505789 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505792 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505795 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505798 2574 flags.go:64] FLAG: --kube-reserved="" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505801 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505804 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505806 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505809 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505812 2574 flags.go:64] FLAG: --lock-file="" Jul 07 12:42:33.055050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505815 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505818 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505821 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505831 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505834 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505837 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505840 2574 flags.go:64] FLAG: --logging-format="text" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505843 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505846 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505849 2574 flags.go:64] FLAG: --manifest-url="" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505852 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505856 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505859 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505863 2574 flags.go:64] FLAG: --max-pods="110" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505866 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505869 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505873 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505876 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505879 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505882 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505885 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505894 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505897 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505900 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 12:42:33.056097 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505903 2574 flags.go:64] FLAG: --pod-cidr="" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505906 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505911 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505913 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505916 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505919 2574 flags.go:64] FLAG: --port="10250" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505922 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505925 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ccc1f858b2034300" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505928 2574 flags.go:64] FLAG: --qos-reserved="" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505931 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505935 2574 flags.go:64] FLAG: --register-node="true" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505938 2574 flags.go:64] FLAG: --register-with-taints="" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505941 2574 flags.go:64] FLAG: --registry-burst="10" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505945 2574 flags.go:64] FLAG: --registry-qps="5" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505962 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505966 2574 flags.go:64] FLAG: --reserved-memory="" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505970 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505973 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505975 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505979 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505982 2574 flags.go:64] FLAG: --runonce="false" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505984 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505987 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505990 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505994 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 12:42:33.056998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.505997 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506000 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506003 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506006 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506009 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506014 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506017 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506020 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506023 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506026 2574 flags.go:64] FLAG: --system-cgroups="" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506029 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506034 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506037 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506040 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506043 2574 flags.go:64] FLAG: --tls-min-version="" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506046 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506049 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506052 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506054 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506057 2574 flags.go:64] FLAG: --v="2" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506061 2574 flags.go:64] FLAG: --version="false" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506066 2574 flags.go:64] FLAG: --vmodule="" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506070 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.506073 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506362 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:33.058082 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506366 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506369 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506372 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506375 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506378 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506380 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506385 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506388 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506390 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506393 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506396 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506398 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506402 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506405 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506407 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506410 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506413 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506416 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506418 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506421 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:33.058934 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506423 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506426 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506428 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506431 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506433 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506438 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506441 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506445 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506448 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506452 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506454 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506457 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506459 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506462 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506464 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506467 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506469 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506472 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506476 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:33.059560 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506479 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506481 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506483 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506486 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506488 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506492 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506495 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506498 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506500 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506503 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506506 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506508 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506510 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506513 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506515 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506518 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506520 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506523 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506526 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506528 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:33.060196 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506531 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506535 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506538 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506542 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506544 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506547 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506549 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506552 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506555 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506557 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506560 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506563 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506566 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506569 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506571 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506574 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506576 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506580 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506583 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506585 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:33.061047 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506588 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506591 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506593 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506596 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506598 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506601 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506603 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506606 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506608 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506611 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506613 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506616 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506618 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506621 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506624 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506626 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506629 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506632 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:33.061679 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:31.506634 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:31.507203 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.898707 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.898726 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.899718 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.899730 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899799 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899804 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899807 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899810 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899813 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899815 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899818 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899821 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899823 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:33.062324 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899826 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899829 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899832 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899834 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899837 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899840 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899842 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899845 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899848 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899850 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899853 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899856 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899859 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899862 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899864 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899867 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899870 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899875 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899879 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:33.062917 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899882 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899886 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899889 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899891 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899894 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899897 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899899 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899902 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899904 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899907 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899910 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899912 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899915 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899917 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899920 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899923 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899925 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899928 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899931 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899933 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:33.063530 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899936 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899938 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899941 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899944 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899947 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899949 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899952 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899956 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899959 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899963 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899966 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899969 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899972 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899975 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899979 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899981 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899984 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899987 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899990 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:33.064147 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899992 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899995 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.899998 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900001 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900004 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900006 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900009 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900016 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900018 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900021 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900024 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900026 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900030 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900034 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900037 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900039 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900042 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900044 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900047 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900050 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:33.064980 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900052 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900056 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900059 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900061 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900064 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900066 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900069 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900072 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900075 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900078 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900081 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900083 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.900089 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900221 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900225 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:33.065558 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900229 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900232 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900236 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900239 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900242 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900246 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900249 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900251 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900254 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900257 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900259 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900262 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900265 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900267 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900270 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900272 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900275 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900277 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900281 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:33.066006 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900284 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900286 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900289 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900291 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900294 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900296 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900299 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900302 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900304 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900307 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900309 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900312 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900314 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900317 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900319 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900322 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900325 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900327 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900332 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:33.066644 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900335 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900338 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900340 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900343 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900346 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900348 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900351 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900354 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900356 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900359 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900361 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900363 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900367 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900370 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900372 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900375 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900377 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900380 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900383 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:33.069153 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900386 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900388 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900391 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900394 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900397 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900399 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900402 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900404 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900406 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900409 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900411 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900414 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900418 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900420 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900423 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900425 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900428 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900430 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900433 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900435 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900438 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:33.069979 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900441 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900443 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900446 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900448 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900452 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900456 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900459 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900462 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900465 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900468 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900471 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900473 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900476 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900478 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900482 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900485 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900487 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900490 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:33.181518 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:32.900493 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.900499 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.900626 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.902837 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.904420 2574 server.go:1019] "Starting client certificate rotation" Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.904520 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.904549 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.907662 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.910618 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.928318 2574 log.go:25] "Validated CRI v1 runtime API" Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.930941 2574 log.go:25] "Validated CRI v1 image API" Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.932650 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 12:42:33.203509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.935382 2574 fs.go:135] Filesystem UUIDs: map[4e51ce9f-4728-4b9c-a2e5-170e80e8cf25:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 fd8ccddf-e131-43bb-bb40-19519be16703:/dev/nvme0n1p3] Jul 07 12:42:33.204142 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.935399 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 12:42:33.204142 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.940531 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.941141 2574 manager.go:216] Machine: {Timestamp:2026-07-07 12:42:32.939601462 +0000 UTC m=+0.383644654 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3079989 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2547ef4a488c79f5b5703719052d10 SystemUUID:ec2547ef-4a48-8c79-f5b5-703719052d10 BootID:e7112544-cbcf-41f0-b52f-fe24067c21d6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9a:af:58:9b:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9a:af:58:9b:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:76:13:2b:26:0d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.941281 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.941434 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.943803 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.943824 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-212.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.944588 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.944597 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.944605 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.945280 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.946470 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.949467 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.949590 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.949617 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.949626 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.949636 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.949650 2574 kubelet.go:398] "Adding apiserver pod source" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.949661 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 12:42:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.950135 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.950152 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.952223 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.954247 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.954292 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955602 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955616 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955621 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955629 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955659 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955666 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955674 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955679 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955686 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955709 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955723 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955729 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955750 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.955756 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.959052 2574 server.go:1268] "Started kubelet" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.959124 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.959239 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.959788 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.960578 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-212.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:32.960640 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:32.960769 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-212.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.960800 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.966601 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:32.967803 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.970794 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:32.970071 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-212.ec2.internal.18c00274c38dc4d5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-212.ec2.internal,UID:ip-10-0-135-212.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-212.ec2.internal,},FirstTimestamp:2026-07-07 12:42:32.959026389 +0000 UTC m=+0.403069627,LastTimestamp:2026-07-07 12:42:32.959026389 +0000 UTC m=+0.403069627,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-212.ec2.internal,}" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.971108 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.971906 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t4dj2" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:32.971966 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-212.ec2.internal\" not found" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.972488 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.972497 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.972722 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.972778 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.972784 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.973579 2574 factory.go:55] Registering systemd factory Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.973594 2574 factory.go:223] Registration of the systemd container factory successfully Jul 07 12:42:33.205155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.973799 2574 factory.go:153] Registering CRI-O factory Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.973810 2574 factory.go:223] Registration of the crio container factory successfully Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.973854 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.973878 2574 factory.go:103] Registering Raw factory Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.973889 2574 manager.go:1196] Started watching for new ooms in manager Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.974197 2574 manager.go:349] Starting recovery of all containers Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.980231 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t4dj2" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:32.980246 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-212.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:32.980319 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.982670 2574 manager.go:354] Recovery completed Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.990869 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.992655 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.992683 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.993034 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.993527 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.993541 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.993546 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.993563 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.994291 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.994316 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.994330 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.994816 2574 policy_none.go:49] "None policy: Start" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.995313 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.995326 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:32.996400 2574 policy_none.go:47] "Start" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:32.996351 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-212.ec2.internal.18c00274c58f2039 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-212.ec2.internal,UID:ip-10-0-135-212.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-212.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-212.ec2.internal,},FirstTimestamp:2026-07-07 12:42:32.992669753 +0000 UTC m=+0.436712928,LastTimestamp:2026-07-07 12:42:32.992669753 +0000 UTC m=+0.436712928,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-212.ec2.internal,}" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.048242 2574 manager.go:337] "Starting Device Plugin manager" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.048381 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.048390 2574 server.go:87] "Starting device plugin registration server" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.048591 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.048600 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.048719 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.048801 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.048810 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.049303 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.049352 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-212.ec2.internal\" not found" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.148911 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.149832 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:33.206288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.149855 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:33.207371 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.149865 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:33.207371 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.149890 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.207371 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.155202 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.207371 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.155222 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-212.ec2.internal\": node \"ip-10-0-135-212.ec2.internal\" not found" Jul 07 12:42:33.207371 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.171011 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-212.ec2.internal\" not found" Jul 07 12:42:33.207371 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.181169 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 12:42:33.207371 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.182429 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 12:42:33.207371 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.182461 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 12:42:33.207371 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.182482 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 12:42:33.207371 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.182510 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 12:42:33.207371 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.194133 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.271744 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-212.ec2.internal\" not found" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.286459 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-212.ec2.internal"] Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.286524 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.288170 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.288199 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.288214 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.290263 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.290372 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.290399 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.291395 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.291420 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.291431 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.291449 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.291463 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.291435 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.293267 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.293289 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.303922 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.303945 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:33.314052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.303954 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:33.407173 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.354959 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-212.ec2.internal\" not found" node="ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.407173 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.359278 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-212.ec2.internal\" not found" node="ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.407173 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.372033 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-212.ec2.internal\" not found" Jul 07 12:42:33.407173 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.382912 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e763d3c31083b98290518ed2246097cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal\" (UID: \"e763d3c31083b98290518ed2246097cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.407173 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.382935 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e763d3c31083b98290518ed2246097cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal\" (UID: \"e763d3c31083b98290518ed2246097cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.407173 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.382950 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/37ae1767e02e91e0c33f5647bfe3b14c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-212.ec2.internal\" (UID: \"37ae1767e02e91e0c33f5647bfe3b14c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.498897 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.472554 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-212.ec2.internal\" not found" Jul 07 12:42:33.498897 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.483843 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e763d3c31083b98290518ed2246097cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal\" (UID: \"e763d3c31083b98290518ed2246097cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.498897 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.483875 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e763d3c31083b98290518ed2246097cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal\" (UID: \"e763d3c31083b98290518ed2246097cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.498897 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.483894 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/37ae1767e02e91e0c33f5647bfe3b14c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-212.ec2.internal\" (UID: \"37ae1767e02e91e0c33f5647bfe3b14c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.498897 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.483956 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e763d3c31083b98290518ed2246097cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal\" (UID: \"e763d3c31083b98290518ed2246097cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.498897 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.484000 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/37ae1767e02e91e0c33f5647bfe3b14c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-212.ec2.internal\" (UID: \"37ae1767e02e91e0c33f5647bfe3b14c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.498897 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.484031 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e763d3c31083b98290518ed2246097cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal\" (UID: \"e763d3c31083b98290518ed2246097cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.573867 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.573010 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-212.ec2.internal\" not found" Jul 07 12:42:33.664885 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.659118 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.664885 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.662527 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.738279 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.673232 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-212.ec2.internal\" not found" Jul 07 12:42:33.820704 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.773569 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-212.ec2.internal\" not found" Jul 07 12:42:33.921899 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.855211 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:42:33.921899 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.878427 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.921899 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.889028 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 12:42:33.921899 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.890853 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-212.ec2.internal" Jul 07 12:42:33.921899 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.896660 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 12:42:33.921899 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.905813 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 12:42:33.921899 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.905938 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 12:42:33.921899 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.905975 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.950106 2574 apiserver.go:52] "Watching apiserver" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.967090 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.968354 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-cbl98","kube-system/konnectivity-agent-2dtxn","openshift-cluster-node-tuning-operator/tuned-xj9qm","openshift-dns/node-resolver-6f6bh","openshift-image-registry/node-ca-qrr6k","openshift-multus/network-metrics-daemon-b4bc4","kube-system/kube-apiserver-proxy-ip-10-0-135-212.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal","openshift-multus/multus-additional-cni-plugins-hgkrq","openshift-multus/multus-fbpcz","openshift-network-diagnostics/network-check-target-w4grp","openshift-network-operator/iptables-alerter-5pzkl","openshift-ovn-kubernetes/ovnkube-node-kl9px"] Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.971326 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.971375 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.972678 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.973303 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.976854 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.976888 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.977008 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wv4kb\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.977122 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.977321 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6f6bh" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.978763 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qrr6k" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.979314 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.979528 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.979653 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-q8jv9\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.979876 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.980489 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.980538 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.980824 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.981031 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9rv6v\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.981189 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.981377 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2hx2h\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.981506 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.981653 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.982705 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.984728 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.984910 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985102 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nj27s\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985247 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 12:42:34.009444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985385 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985863 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-sysconfig\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985880 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-kubernetes\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985895 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-sys\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985908 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-host\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985922 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-tmp\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985937 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-sysctl-d\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985956 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl5tc\" (UniqueName: \"kubernetes.io/projected/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-kube-api-access-rl5tc\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985971 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-kubelet-config\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.985992 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-modprobe-d\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.986006 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-sysctl-conf\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.986019 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-run\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.986032 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-lib-modules\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.986045 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-tuned\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.986067 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.986089 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cc610f8c-e866-4ff6-8a27-172ddb224a0c-konnectivity-ca\") pod \"konnectivity-agent-2dtxn\" (UID: \"cc610f8c-e866-4ff6-8a27-172ddb224a0c\") " pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.986102 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-systemd\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.986117 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-var-lib-kubelet\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.986132 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-dbus\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.986145 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cc610f8c-e866-4ff6-8a27-172ddb224a0c-agent-certs\") pod \"konnectivity-agent-2dtxn\" (UID: \"cc610f8c-e866-4ff6-8a27-172ddb224a0c\") " pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.989010 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.989129 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2q22v\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.989259 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.989392 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.989547 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.989652 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.989793 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.989932 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:33.989996 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.990032 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.990276 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 12:37:32 +0000 UTC" deadline="2028-04-15 06:23:30.73916019 +0000 UTC" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.990290 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15545h40m56.748871888s" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.991241 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5pzkl" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.991980 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.992317 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g5jtt\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.993171 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.993474 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.993592 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.993769 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.993891 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qlxnj\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.996080 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.996149 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gfj2n\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.996170 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.996259 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.996446 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.997443 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:33.997500 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.006127 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4kq74" Jul 07 12:42:34.012844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.012196 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4kq74" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.037587 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.076050 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086236 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-var-lib-kubelet\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086261 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-cni-dir\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086278 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-os-release\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086293 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-slash\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086327 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-sysconfig\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086354 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-system-cni-dir\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086367 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-var-lib-kubelet\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086378 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-hostroot\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086397 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-sys-fs\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086411 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-sysconfig\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086425 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60f064c6-f290-4194-8c69-9f0c1712498e-ovnkube-script-lib\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086441 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-run-multus-certs\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086464 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae03df12-73df-481b-b330-c624c016e8cb-tmp-dir\") pod \"node-resolver-6f6bh\" (UID: \"ae03df12-73df-481b-b330-c624c016e8cb\") " pod="openshift-dns/node-resolver-6f6bh" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086481 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlrmd\" (UniqueName: \"kubernetes.io/projected/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-kube-api-access-xlrmd\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086543 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-systemd-units\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.099020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086568 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-run-openvswitch\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086584 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60f064c6-f290-4194-8c69-9f0c1712498e-ovn-node-metrics-cert\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086607 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-system-cni-dir\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086633 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-run\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086709 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-tuned\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086738 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-run\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086757 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-var-lib-cni-multus\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086813 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-etc-kubernetes\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086832 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-cni-bin\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086847 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-cni-netd\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086869 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-cnibin\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cc610f8c-e866-4ff6-8a27-172ddb224a0c-konnectivity-ca\") pod \"konnectivity-agent-2dtxn\" (UID: \"cc610f8c-e866-4ff6-8a27-172ddb224a0c\") " pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-systemd\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086922 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-var-lib-kubelet\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086940 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be21f262-be06-421e-bb1e-f74edd4fffef-host-slash\") pod \"iptables-alerter-5pzkl\" (UID: \"be21f262-be06-421e-bb1e-f74edd4fffef\") " pod="openshift-network-operator/iptables-alerter-5pzkl" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086971 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqzdl\" (UniqueName: \"kubernetes.io/projected/ae03df12-73df-481b-b330-c624c016e8cb-kube-api-access-tqzdl\") pod \"node-resolver-6f6bh\" (UID: \"ae03df12-73df-481b-b330-c624c016e8cb\") " pod="openshift-dns/node-resolver-6f6bh" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086986 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-systemd\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.100244 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.086994 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-kubelet\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087020 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-dbus\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087042 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-kubernetes\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087058 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-socket-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087073 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-cnibin\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087085 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-dbus\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087113 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-kubernetes\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087155 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-node-log\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087171 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-run-ovn-kubernetes\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087201 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60f064c6-f290-4194-8c69-9f0c1712498e-ovnkube-config\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087219 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fshrg\" (UniqueName: \"kubernetes.io/projected/10f829cd-c046-4df6-a5d6-ceff68b1656a-kube-api-access-fshrg\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087239 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-modprobe-d\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087271 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-sysctl-conf\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087294 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-registration-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087309 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-run-systemd\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087325 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087329 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-modprobe-d\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.101146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087350 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-run-k8s-cni-cncf-io\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087382 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be21f262-be06-421e-bb1e-f74edd4fffef-iptables-alerter-script\") pod \"iptables-alerter-5pzkl\" (UID: \"be21f262-be06-421e-bb1e-f74edd4fffef\") " pod="openshift-network-operator/iptables-alerter-5pzkl" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087403 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cc610f8c-e866-4ff6-8a27-172ddb224a0c-agent-certs\") pod \"konnectivity-agent-2dtxn\" (UID: \"cc610f8c-e866-4ff6-8a27-172ddb224a0c\") " pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087428 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-sysctl-conf\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.087480 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087490 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-host\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.087548 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret podName:5e3e1d99-1fd1-4f01-980b-146f14cb0b43 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:34.587528226 +0000 UTC m=+2.031571387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret") pod "global-pull-secret-syncer-cbl98" (UID: "5e3e1d99-1fd1-4f01-980b-146f14cb0b43") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087572 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-tmp\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087599 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-host\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087630 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-daemon-config\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087656 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2jp2\" (UniqueName: \"kubernetes.io/projected/be21f262-be06-421e-bb1e-f74edd4fffef-kube-api-access-j2jp2\") pod \"iptables-alerter-5pzkl\" (UID: \"be21f262-be06-421e-bb1e-f74edd4fffef\") " pod="openshift-network-operator/iptables-alerter-5pzkl" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087675 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-device-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087719 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-run-ovn\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087743 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-sysctl-d\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087840 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl5tc\" (UniqueName: \"kubernetes.io/projected/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-kube-api-access-rl5tc\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087883 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-sysctl-d\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.102388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087930 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-socket-dir-parent\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.087981 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-conf-dir\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088003 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-run-netns\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088017 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-etc-openvswitch\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088048 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-log-socket\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088100 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a67391a-c8bc-4eaf-a00a-c728f059574c-serviceca\") pod \"node-ca-qrr6k\" (UID: \"3a67391a-c8bc-4eaf-a00a-c728f059574c\") " pod="openshift-image-registry/node-ca-qrr6k" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088163 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-kubelet-config\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088209 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-lib-modules\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088232 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-var-lib-cni-bin\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088264 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60f064c6-f290-4194-8c69-9f0c1712498e-env-overrides\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088284 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-kubelet-config\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088313 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-os-release\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088320 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-lib-modules\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088338 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87df0b3d-276d-4158-942c-4bdb9d08732d-cni-binary-copy\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088385 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088411 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-var-lib-openvswitch\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088433 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.103036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088458 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a67391a-c8bc-4eaf-a00a-c728f059574c-host\") pod \"node-ca-qrr6k\" (UID: \"3a67391a-c8bc-4eaf-a00a-c728f059574c\") " pod="openshift-image-registry/node-ca-qrr6k" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088482 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/10f829cd-c046-4df6-a5d6-ceff68b1656a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088506 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-sys\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088528 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ae03df12-73df-481b-b330-c624c016e8cb-hosts-file\") pod \"node-resolver-6f6bh\" (UID: \"ae03df12-73df-481b-b330-c624c016e8cb\") " pod="openshift-dns/node-resolver-6f6bh" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088584 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmrvt\" (UniqueName: \"kubernetes.io/projected/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-kube-api-access-gmrvt\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088616 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wcr5\" (UniqueName: \"kubernetes.io/projected/60f064c6-f290-4194-8c69-9f0c1712498e-kube-api-access-7wcr5\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088606 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-sys\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088636 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4fs9\" (UniqueName: \"kubernetes.io/projected/3a67391a-c8bc-4eaf-a00a-c728f059574c-kube-api-access-z4fs9\") pod \"node-ca-qrr6k\" (UID: \"3a67391a-c8bc-4eaf-a00a-c728f059574c\") " pod="openshift-image-registry/node-ca-qrr6k" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088672 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10f829cd-c046-4df6-a5d6-ceff68b1656a-cni-binary-copy\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088712 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-run-netns\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088744 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwsgx\" (UniqueName: \"kubernetes.io/projected/87df0b3d-276d-4158-942c-4bdb9d08732d-kube-api-access-jwsgx\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088763 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pghr\" (UniqueName: \"kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr\") pod \"network-check-target-w4grp\" (UID: \"ba088306-b1bd-4a02-9ee2-e547cc187b9f\") " pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088797 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088833 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-etc-selinux\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.088865 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/10f829cd-c046-4df6-a5d6-ceff68b1656a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.089088 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 12:42:34.104283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.093069 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cc610f8c-e866-4ff6-8a27-172ddb224a0c-agent-certs\") pod \"konnectivity-agent-2dtxn\" (UID: \"cc610f8c-e866-4ff6-8a27-172ddb224a0c\") " pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:34.110779 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.110753 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cc610f8c-e866-4ff6-8a27-172ddb224a0c-konnectivity-ca\") pod \"konnectivity-agent-2dtxn\" (UID: \"cc610f8c-e866-4ff6-8a27-172ddb224a0c\") " pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:34.111141 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.111117 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-tmp\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.111393 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.111373 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl5tc\" (UniqueName: \"kubernetes.io/projected/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-kube-api-access-rl5tc\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.112238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.112216 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5ff2cc97-4b16-4c08-a180-4b8341bca5e9-etc-tuned\") pod \"tuned-xj9qm\" (UID: \"5ff2cc97-4b16-4c08-a180-4b8341bca5e9\") " pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.189122 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189090 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87df0b3d-276d-4158-942c-4bdb9d08732d-cni-binary-copy\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.189342 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189131 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:34.189342 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189158 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-var-lib-openvswitch\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.189342 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189186 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.189342 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189213 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a67391a-c8bc-4eaf-a00a-c728f059574c-host\") pod \"node-ca-qrr6k\" (UID: \"3a67391a-c8bc-4eaf-a00a-c728f059574c\") " pod="openshift-image-registry/node-ca-qrr6k" Jul 07 12:42:34.189342 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.189242 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:34.189342 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189266 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-var-lib-openvswitch\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.189342 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.189312 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs podName:9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:34.689293502 +0000 UTC m=+2.133336661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs") pod "network-metrics-daemon-b4bc4" (UID: "9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:34.189342 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189326 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.189342 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189243 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/10f829cd-c046-4df6-a5d6-ceff68b1656a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189577 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a67391a-c8bc-4eaf-a00a-c728f059574c-host\") pod \"node-ca-qrr6k\" (UID: \"3a67391a-c8bc-4eaf-a00a-c728f059574c\") " pod="openshift-image-registry/node-ca-qrr6k" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189638 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ae03df12-73df-481b-b330-c624c016e8cb-hosts-file\") pod \"node-resolver-6f6bh\" (UID: \"ae03df12-73df-481b-b330-c624c016e8cb\") " pod="openshift-dns/node-resolver-6f6bh" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189666 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmrvt\" (UniqueName: \"kubernetes.io/projected/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-kube-api-access-gmrvt\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189724 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wcr5\" (UniqueName: \"kubernetes.io/projected/60f064c6-f290-4194-8c69-9f0c1712498e-kube-api-access-7wcr5\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189744 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ae03df12-73df-481b-b330-c624c016e8cb-hosts-file\") pod \"node-resolver-6f6bh\" (UID: \"ae03df12-73df-481b-b330-c624c016e8cb\") " pod="openshift-dns/node-resolver-6f6bh" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189857 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/10f829cd-c046-4df6-a5d6-ceff68b1656a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189884 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4fs9\" (UniqueName: \"kubernetes.io/projected/3a67391a-c8bc-4eaf-a00a-c728f059574c-kube-api-access-z4fs9\") pod \"node-ca-qrr6k\" (UID: \"3a67391a-c8bc-4eaf-a00a-c728f059574c\") " pod="openshift-image-registry/node-ca-qrr6k" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189923 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10f829cd-c046-4df6-a5d6-ceff68b1656a-cni-binary-copy\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189941 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87df0b3d-276d-4158-942c-4bdb9d08732d-cni-binary-copy\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189953 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-run-netns\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.189979 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwsgx\" (UniqueName: \"kubernetes.io/projected/87df0b3d-276d-4158-942c-4bdb9d08732d-kube-api-access-jwsgx\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190005 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pghr\" (UniqueName: \"kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr\") pod \"network-check-target-w4grp\" (UID: \"ba088306-b1bd-4a02-9ee2-e547cc187b9f\") " pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190028 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190053 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-run-netns\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190093 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190174 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-etc-selinux\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.190341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190209 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/10f829cd-c046-4df6-a5d6-ceff68b1656a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190238 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-cni-dir\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190263 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-os-release\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190288 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-etc-selinux\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190324 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-cni-dir\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190302 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-slash\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190366 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-os-release\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190369 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-system-cni-dir\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190377 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-slash\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190364 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10f829cd-c046-4df6-a5d6-ceff68b1656a-cni-binary-copy\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190409 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-hostroot\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190417 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-system-cni-dir\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190435 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-sys-fs\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190443 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-hostroot\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190463 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60f064c6-f290-4194-8c69-9f0c1712498e-ovnkube-script-lib\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190495 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-run-multus-certs\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190520 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae03df12-73df-481b-b330-c624c016e8cb-tmp-dir\") pod \"node-resolver-6f6bh\" (UID: \"ae03df12-73df-481b-b330-c624c016e8cb\") " pod="openshift-dns/node-resolver-6f6bh" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190548 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-run-multus-certs\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.191238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190551 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlrmd\" (UniqueName: \"kubernetes.io/projected/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-kube-api-access-xlrmd\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190579 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-systemd-units\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190606 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-run-openvswitch\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190613 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-sys-fs\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190631 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60f064c6-f290-4194-8c69-9f0c1712498e-ovn-node-metrics-cert\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190660 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-system-cni-dir\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190710 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-var-lib-cni-multus\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190737 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-etc-kubernetes\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190769 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-cni-bin\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190766 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/10f829cd-c046-4df6-a5d6-ceff68b1656a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190786 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-run-openvswitch\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190830 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-cni-netd\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190838 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-systemd-units\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190851 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-var-lib-cni-multus\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190865 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-cnibin\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190895 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-etc-kubernetes\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190896 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-system-cni-dir\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.192192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190895 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-var-lib-kubelet\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190936 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-var-lib-kubelet\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190943 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-cni-bin\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190941 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-cni-netd\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190958 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be21f262-be06-421e-bb1e-f74edd4fffef-host-slash\") pod \"iptables-alerter-5pzkl\" (UID: \"be21f262-be06-421e-bb1e-f74edd4fffef\") " pod="openshift-network-operator/iptables-alerter-5pzkl" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190985 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqzdl\" (UniqueName: \"kubernetes.io/projected/ae03df12-73df-481b-b330-c624c016e8cb-kube-api-access-tqzdl\") pod \"node-resolver-6f6bh\" (UID: \"ae03df12-73df-481b-b330-c624c016e8cb\") " pod="openshift-dns/node-resolver-6f6bh" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190992 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be21f262-be06-421e-bb1e-f74edd4fffef-host-slash\") pod \"iptables-alerter-5pzkl\" (UID: \"be21f262-be06-421e-bb1e-f74edd4fffef\") " pod="openshift-network-operator/iptables-alerter-5pzkl" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.190987 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-cnibin\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191007 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-kubelet\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191034 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-kubelet\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191049 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-socket-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191078 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-cnibin\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191103 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-node-log\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191124 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60f064c6-f290-4194-8c69-9f0c1712498e-ovnkube-script-lib\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191130 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-run-ovn-kubernetes\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191141 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-node-log\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191160 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60f064c6-f290-4194-8c69-9f0c1712498e-ovnkube-config\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191184 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-cnibin\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.192812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191198 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fshrg\" (UniqueName: \"kubernetes.io/projected/10f829cd-c046-4df6-a5d6-ceff68b1656a-kube-api-access-fshrg\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191205 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-socket-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191224 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-run-ovn-kubernetes\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191226 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-registration-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191263 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-registration-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191266 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-run-systemd\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191303 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-run-k8s-cni-cncf-io\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191343 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be21f262-be06-421e-bb1e-f74edd4fffef-iptables-alerter-script\") pod \"iptables-alerter-5pzkl\" (UID: \"be21f262-be06-421e-bb1e-f74edd4fffef\") " pod="openshift-network-operator/iptables-alerter-5pzkl" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191334 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-run-systemd\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191371 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-daemon-config\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191374 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-run-k8s-cni-cncf-io\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2jp2\" (UniqueName: \"kubernetes.io/projected/be21f262-be06-421e-bb1e-f74edd4fffef-kube-api-access-j2jp2\") pod \"iptables-alerter-5pzkl\" (UID: \"be21f262-be06-421e-bb1e-f74edd4fffef\") " pod="openshift-network-operator/iptables-alerter-5pzkl" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191437 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-device-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191462 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-run-ovn\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191484 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191607 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-socket-dir-parent\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191633 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-run-ovn\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191635 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-conf-dir\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191666 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60f064c6-f290-4194-8c69-9f0c1712498e-ovnkube-config\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191679 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-conf-dir\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191707 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-run-netns\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191741 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-host-run-netns\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191774 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-socket-dir-parent\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191796 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-device-dir\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191827 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-etc-openvswitch\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191852 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-log-socket\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191874 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a67391a-c8bc-4eaf-a00a-c728f059574c-serviceca\") pod \"node-ca-qrr6k\" (UID: \"3a67391a-c8bc-4eaf-a00a-c728f059574c\") " pod="openshift-image-registry/node-ca-qrr6k" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191881 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87df0b3d-276d-4158-942c-4bdb9d08732d-multus-daemon-config\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191901 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-var-lib-cni-bin\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191925 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60f064c6-f290-4194-8c69-9f0c1712498e-env-overrides\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191935 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-etc-openvswitch\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191947 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87df0b3d-276d-4158-942c-4bdb9d08732d-host-var-lib-cni-bin\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191903 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191947 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-os-release\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.191989 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60f064c6-f290-4194-8c69-9f0c1712498e-log-socket\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.193855 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.192047 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10f829cd-c046-4df6-a5d6-ceff68b1656a-os-release\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.194346 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.192201 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae03df12-73df-481b-b330-c624c016e8cb-tmp-dir\") pod \"node-resolver-6f6bh\" (UID: \"ae03df12-73df-481b-b330-c624c016e8cb\") " pod="openshift-dns/node-resolver-6f6bh" Jul 07 12:42:34.194346 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.192222 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be21f262-be06-421e-bb1e-f74edd4fffef-iptables-alerter-script\") pod \"iptables-alerter-5pzkl\" (UID: \"be21f262-be06-421e-bb1e-f74edd4fffef\") " pod="openshift-network-operator/iptables-alerter-5pzkl" Jul 07 12:42:34.194346 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.192326 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a67391a-c8bc-4eaf-a00a-c728f059574c-serviceca\") pod \"node-ca-qrr6k\" (UID: \"3a67391a-c8bc-4eaf-a00a-c728f059574c\") " pod="openshift-image-registry/node-ca-qrr6k" Jul 07 12:42:34.194346 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.192374 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60f064c6-f290-4194-8c69-9f0c1712498e-env-overrides\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.194346 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.193054 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60f064c6-f290-4194-8c69-9f0c1712498e-ovn-node-metrics-cert\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.200269 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.200069 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:34.200269 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.200094 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:34.200269 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.200108 2574 projected.go:196] Error preparing data for projected volume kube-api-access-7pghr for pod openshift-network-diagnostics/network-check-target-w4grp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:34.200269 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.200170 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr podName:ba088306-b1bd-4a02-9ee2-e547cc187b9f nodeName:}" failed. No retries permitted until 2026-07-07 12:42:34.700154048 +0000 UTC m=+2.144197206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7pghr" (UniqueName: "kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr") pod "network-check-target-w4grp" (UID: "ba088306-b1bd-4a02-9ee2-e547cc187b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:34.201345 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.200942 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wcr5\" (UniqueName: \"kubernetes.io/projected/60f064c6-f290-4194-8c69-9f0c1712498e-kube-api-access-7wcr5\") pod \"ovnkube-node-kl9px\" (UID: \"60f064c6-f290-4194-8c69-9f0c1712498e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.203048 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.203029 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwsgx\" (UniqueName: \"kubernetes.io/projected/87df0b3d-276d-4158-942c-4bdb9d08732d-kube-api-access-jwsgx\") pod \"multus-fbpcz\" (UID: \"87df0b3d-276d-4158-942c-4bdb9d08732d\") " pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.203146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.203105 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqzdl\" (UniqueName: \"kubernetes.io/projected/ae03df12-73df-481b-b330-c624c016e8cb-kube-api-access-tqzdl\") pod \"node-resolver-6f6bh\" (UID: \"ae03df12-73df-481b-b330-c624c016e8cb\") " pod="openshift-dns/node-resolver-6f6bh" Jul 07 12:42:34.203146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.203117 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4fs9\" (UniqueName: \"kubernetes.io/projected/3a67391a-c8bc-4eaf-a00a-c728f059574c-kube-api-access-z4fs9\") pod \"node-ca-qrr6k\" (UID: \"3a67391a-c8bc-4eaf-a00a-c728f059574c\") " pod="openshift-image-registry/node-ca-qrr6k" Jul 07 12:42:34.203412 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.203396 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlrmd\" (UniqueName: \"kubernetes.io/projected/2b81ef9a-ef17-4f72-a92a-d81a99fc3b02-kube-api-access-xlrmd\") pod \"aws-ebs-csi-driver-node-s6vqk\" (UID: \"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.203611 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.203591 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2jp2\" (UniqueName: \"kubernetes.io/projected/be21f262-be06-421e-bb1e-f74edd4fffef-kube-api-access-j2jp2\") pod \"iptables-alerter-5pzkl\" (UID: \"be21f262-be06-421e-bb1e-f74edd4fffef\") " pod="openshift-network-operator/iptables-alerter-5pzkl" Jul 07 12:42:34.204147 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.204131 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmrvt\" (UniqueName: \"kubernetes.io/projected/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-kube-api-access-gmrvt\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:34.204793 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.204772 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fshrg\" (UniqueName: \"kubernetes.io/projected/10f829cd-c046-4df6-a5d6-ceff68b1656a-kube-api-access-fshrg\") pod \"multus-additional-cni-plugins-hgkrq\" (UID: \"10f829cd-c046-4df6-a5d6-ceff68b1656a\") " pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.221983 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.221961 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5pzkl" Jul 07 12:42:34.227818 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.227798 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:42:34.317258 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:34.317220 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ae1767e02e91e0c33f5647bfe3b14c.slice/crio-3449fa93088e0cfc41962267800a66c78f455e45eaa13cc956a28d581cbf412e WatchSource:0}: Error finding container 3449fa93088e0cfc41962267800a66c78f455e45eaa13cc956a28d581cbf412e: Status 404 returned error can't find the container with id 3449fa93088e0cfc41962267800a66c78f455e45eaa13cc956a28d581cbf412e Jul 07 12:42:34.317579 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:34.317556 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode763d3c31083b98290518ed2246097cf.slice/crio-db6ce31ecfc1ce2556a54533cf28919e2e9ed2fbfecf67183197df35dbd6d0ff WatchSource:0}: Error finding container db6ce31ecfc1ce2556a54533cf28919e2e9ed2fbfecf67183197df35dbd6d0ff: Status 404 returned error can't find the container with id db6ce31ecfc1ce2556a54533cf28919e2e9ed2fbfecf67183197df35dbd6d0ff Jul 07 12:42:34.321045 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.321030 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 12:42:34.357151 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.357100 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:42:34.366137 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.366122 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:34.375062 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:34.375025 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc610f8c_e866_4ff6_8a27_172ddb224a0c.slice/crio-b8df38802a64bebc645cbd3d755d20fb1ecb8651ef69fa288b5089862008d032 WatchSource:0}: Error finding container b8df38802a64bebc645cbd3d755d20fb1ecb8651ef69fa288b5089862008d032: Status 404 returned error can't find the container with id b8df38802a64bebc645cbd3d755d20fb1ecb8651ef69fa288b5089862008d032 Jul 07 12:42:34.391805 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.391782 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" Jul 07 12:42:34.399908 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:34.399887 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff2cc97_4b16_4c08_a180_4b8341bca5e9.slice/crio-9a8c4b84e2006b7c97d19eb6270acd32f6348b380b4571fc6cd89fefa5adf0fb WatchSource:0}: Error finding container 9a8c4b84e2006b7c97d19eb6270acd32f6348b380b4571fc6cd89fefa5adf0fb: Status 404 returned error can't find the container with id 9a8c4b84e2006b7c97d19eb6270acd32f6348b380b4571fc6cd89fefa5adf0fb Jul 07 12:42:34.439910 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.439886 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6f6bh" Jul 07 12:42:34.443700 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.443669 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qrr6k" Jul 07 12:42:34.448508 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:34.448486 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae03df12_73df_481b_b330_c624c016e8cb.slice/crio-635281d7673431e0e5de07f05c0efbfe10605e4f012cc63e6ae3c3b7b605a4a7 WatchSource:0}: Error finding container 635281d7673431e0e5de07f05c0efbfe10605e4f012cc63e6ae3c3b7b605a4a7: Status 404 returned error can't find the container with id 635281d7673431e0e5de07f05c0efbfe10605e4f012cc63e6ae3c3b7b605a4a7 Jul 07 12:42:34.452145 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:34.452119 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a67391a_c8bc_4eaf_a00a_c728f059574c.slice/crio-abe0aee772648335a2c6716409c16484c6ecfffc6d52e3f6352ade7e5fd2f275 WatchSource:0}: Error finding container abe0aee772648335a2c6716409c16484c6ecfffc6d52e3f6352ade7e5fd2f275: Status 404 returned error can't find the container with id abe0aee772648335a2c6716409c16484c6ecfffc6d52e3f6352ade7e5fd2f275 Jul 07 12:42:34.481068 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.481045 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" Jul 07 12:42:34.487994 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.487975 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" Jul 07 12:42:34.489589 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:34.489567 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b81ef9a_ef17_4f72_a92a_d81a99fc3b02.slice/crio-3751941954f2c87b7eb0667a8815281f533ebaaccdf5c67bfd5a5bba627bcc31 WatchSource:0}: Error finding container 3751941954f2c87b7eb0667a8815281f533ebaaccdf5c67bfd5a5bba627bcc31: Status 404 returned error can't find the container with id 3751941954f2c87b7eb0667a8815281f533ebaaccdf5c67bfd5a5bba627bcc31 Jul 07 12:42:34.495983 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:34.495964 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10f829cd_c046_4df6_a5d6_ceff68b1656a.slice/crio-e662d9e00c2fa7afd1911a80717847785870f7cd198a4e9d337cd268200d001c WatchSource:0}: Error finding container e662d9e00c2fa7afd1911a80717847785870f7cd198a4e9d337cd268200d001c: Status 404 returned error can't find the container with id e662d9e00c2fa7afd1911a80717847785870f7cd198a4e9d337cd268200d001c Jul 07 12:42:34.502402 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.502385 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fbpcz" Jul 07 12:42:34.510593 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:34.510575 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87df0b3d_276d_4158_942c_4bdb9d08732d.slice/crio-4f5acb722077b914877b79b1c13dc7c666bfe686e788fda8bed92ca7ca7ffe4b WatchSource:0}: Error finding container 4f5acb722077b914877b79b1c13dc7c666bfe686e788fda8bed92ca7ca7ffe4b: Status 404 returned error can't find the container with id 4f5acb722077b914877b79b1c13dc7c666bfe686e788fda8bed92ca7ca7ffe4b Jul 07 12:42:34.589901 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:34.589879 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60f064c6_f290_4194_8c69_9f0c1712498e.slice/crio-941fd742a4ad504c94435a91ea24519cc9ba850ead09cd8e932866f44d4102f3 WatchSource:0}: Error finding container 941fd742a4ad504c94435a91ea24519cc9ba850ead09cd8e932866f44d4102f3: Status 404 returned error can't find the container with id 941fd742a4ad504c94435a91ea24519cc9ba850ead09cd8e932866f44d4102f3 Jul 07 12:42:34.595884 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.595865 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:34.596021 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.596005 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:34.596067 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.596058 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret podName:5e3e1d99-1fd1-4f01-980b-146f14cb0b43 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:35.596043886 +0000 UTC m=+3.040087048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret") pod "global-pull-secret-syncer-cbl98" (UID: "5e3e1d99-1fd1-4f01-980b-146f14cb0b43") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:34.696986 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.696955 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:34.697137 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.697057 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:34.697137 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.697103 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs podName:9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:35.697089496 +0000 UTC m=+3.141132653 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs") pod "network-metrics-daemon-b4bc4" (UID: "9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:34.797559 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.797503 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pghr\" (UniqueName: \"kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr\") pod \"network-check-target-w4grp\" (UID: \"ba088306-b1bd-4a02-9ee2-e547cc187b9f\") " pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:34.797746 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.797673 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:34.797746 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.797716 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:34.797746 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.797730 2574 projected.go:196] Error preparing data for projected volume kube-api-access-7pghr for pod openshift-network-diagnostics/network-check-target-w4grp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:34.797897 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:34.797791 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr podName:ba088306-b1bd-4a02-9ee2-e547cc187b9f nodeName:}" failed. No retries permitted until 2026-07-07 12:42:35.797772372 +0000 UTC m=+3.241815547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pghr" (UniqueName: "kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr") pod "network-check-target-w4grp" (UID: "ba088306-b1bd-4a02-9ee2-e547cc187b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:34.998993 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:34.998919 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:42:35.015137 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.015073 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 12:37:34 +0000 UTC" deadline="2028-03-12 00:02:59.911335728 +0000 UTC" Jul 07 12:42:35.015137 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.015135 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14723h20m24.896204087s" Jul 07 12:42:35.159737 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:42:35.159702 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe21f262_be06_421e_bb1e_f74edd4fffef.slice/crio-99f9c8f630b7865944ecdc833d14900b73e523cd8b6ad54df34a137d5e59fd1c WatchSource:0}: Error finding container 99f9c8f630b7865944ecdc833d14900b73e523cd8b6ad54df34a137d5e59fd1c: Status 404 returned error can't find the container with id 99f9c8f630b7865944ecdc833d14900b73e523cd8b6ad54df34a137d5e59fd1c Jul 07 12:42:35.184612 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.184585 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:35.184813 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:35.184769 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:35.208110 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.208000 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" event={"ID":"10f829cd-c046-4df6-a5d6-ceff68b1656a","Type":"ContainerStarted","Data":"e662d9e00c2fa7afd1911a80717847785870f7cd198a4e9d337cd268200d001c"} Jul 07 12:42:35.216578 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.216513 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" event={"ID":"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02","Type":"ContainerStarted","Data":"3751941954f2c87b7eb0667a8815281f533ebaaccdf5c67bfd5a5bba627bcc31"} Jul 07 12:42:35.223781 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.223751 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qrr6k" event={"ID":"3a67391a-c8bc-4eaf-a00a-c728f059574c","Type":"ContainerStarted","Data":"abe0aee772648335a2c6716409c16484c6ecfffc6d52e3f6352ade7e5fd2f275"} Jul 07 12:42:35.232520 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.232462 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2dtxn" event={"ID":"cc610f8c-e866-4ff6-8a27-172ddb224a0c","Type":"ContainerStarted","Data":"b8df38802a64bebc645cbd3d755d20fb1ecb8651ef69fa288b5089862008d032"} Jul 07 12:42:35.249520 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.249451 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-212.ec2.internal" event={"ID":"37ae1767e02e91e0c33f5647bfe3b14c","Type":"ContainerStarted","Data":"3449fa93088e0cfc41962267800a66c78f455e45eaa13cc956a28d581cbf412e"} Jul 07 12:42:35.252107 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.252077 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fbpcz" event={"ID":"87df0b3d-276d-4158-942c-4bdb9d08732d","Type":"ContainerStarted","Data":"4f5acb722077b914877b79b1c13dc7c666bfe686e788fda8bed92ca7ca7ffe4b"} Jul 07 12:42:35.253875 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.253841 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6f6bh" event={"ID":"ae03df12-73df-481b-b330-c624c016e8cb","Type":"ContainerStarted","Data":"635281d7673431e0e5de07f05c0efbfe10605e4f012cc63e6ae3c3b7b605a4a7"} Jul 07 12:42:35.256187 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.256162 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" event={"ID":"5ff2cc97-4b16-4c08-a180-4b8341bca5e9","Type":"ContainerStarted","Data":"9a8c4b84e2006b7c97d19eb6270acd32f6348b380b4571fc6cd89fefa5adf0fb"} Jul 07 12:42:35.263926 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.263857 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" event={"ID":"e763d3c31083b98290518ed2246097cf","Type":"ContainerStarted","Data":"db6ce31ecfc1ce2556a54533cf28919e2e9ed2fbfecf67183197df35dbd6d0ff"} Jul 07 12:42:35.271348 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.271321 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5pzkl" event={"ID":"be21f262-be06-421e-bb1e-f74edd4fffef","Type":"ContainerStarted","Data":"99f9c8f630b7865944ecdc833d14900b73e523cd8b6ad54df34a137d5e59fd1c"} Jul 07 12:42:35.274020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.273994 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" event={"ID":"60f064c6-f290-4194-8c69-9f0c1712498e","Type":"ContainerStarted","Data":"941fd742a4ad504c94435a91ea24519cc9ba850ead09cd8e932866f44d4102f3"} Jul 07 12:42:35.611373 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.611108 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:35.611373 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:35.611300 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:35.611373 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:35.611359 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret podName:5e3e1d99-1fd1-4f01-980b-146f14cb0b43 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:37.611342096 +0000 UTC m=+5.055385279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret") pod "global-pull-secret-syncer-cbl98" (UID: "5e3e1d99-1fd1-4f01-980b-146f14cb0b43") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:35.712035 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.712000 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:35.712214 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:35.712167 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:35.712280 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:35.712242 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs podName:9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:37.712222484 +0000 UTC m=+5.156265647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs") pod "network-metrics-daemon-b4bc4" (UID: "9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:35.813387 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:35.813254 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pghr\" (UniqueName: \"kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr\") pod \"network-check-target-w4grp\" (UID: \"ba088306-b1bd-4a02-9ee2-e547cc187b9f\") " pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:35.813598 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:35.813429 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:35.813598 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:35.813450 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:35.813598 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:35.813463 2574 projected.go:196] Error preparing data for projected volume kube-api-access-7pghr for pod openshift-network-diagnostics/network-check-target-w4grp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:35.813598 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:35.813531 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr podName:ba088306-b1bd-4a02-9ee2-e547cc187b9f nodeName:}" failed. No retries permitted until 2026-07-07 12:42:37.813504467 +0000 UTC m=+5.257547641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pghr" (UniqueName: "kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr") pod "network-check-target-w4grp" (UID: "ba088306-b1bd-4a02-9ee2-e547cc187b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:36.015997 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:36.015954 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 12:37:34 +0000 UTC" deadline="2027-12-23 14:59:14.710373727 +0000 UTC" Jul 07 12:42:36.015997 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:36.015994 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12818h16m38.694383298s" Jul 07 12:42:36.183675 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:36.183639 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:36.183862 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:36.183803 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:36.184544 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:36.184232 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:36.184544 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:36.184327 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:36.883536 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:36.883430 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:42:37.183861 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:37.183788 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:37.184343 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:37.183924 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:37.630004 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:37.629963 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:37.630199 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:37.630133 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:37.630256 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:37.630198 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret podName:5e3e1d99-1fd1-4f01-980b-146f14cb0b43 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:41.630179849 +0000 UTC m=+9.074223008 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret") pod "global-pull-secret-syncer-cbl98" (UID: "5e3e1d99-1fd1-4f01-980b-146f14cb0b43") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:37.730549 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:37.730511 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:37.730740 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:37.730680 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:37.730832 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:37.730757 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs podName:9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:41.730738928 +0000 UTC m=+9.174782089 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs") pod "network-metrics-daemon-b4bc4" (UID: "9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:37.831731 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:37.831479 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pghr\" (UniqueName: \"kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr\") pod \"network-check-target-w4grp\" (UID: \"ba088306-b1bd-4a02-9ee2-e547cc187b9f\") " pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:37.832447 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:37.831974 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:37.832447 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:37.832004 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:37.832447 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:37.832026 2574 projected.go:196] Error preparing data for projected volume kube-api-access-7pghr for pod openshift-network-diagnostics/network-check-target-w4grp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:37.832447 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:37.832096 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr podName:ba088306-b1bd-4a02-9ee2-e547cc187b9f nodeName:}" failed. No retries permitted until 2026-07-07 12:42:41.832077321 +0000 UTC m=+9.276120493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pghr" (UniqueName: "kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr") pod "network-check-target-w4grp" (UID: "ba088306-b1bd-4a02-9ee2-e547cc187b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:38.183261 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:38.183224 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:38.183420 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:38.183373 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:38.183476 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:38.183429 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:38.183587 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:38.183567 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:39.185995 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:39.185956 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:39.186451 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:39.186095 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:40.183159 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:40.183123 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:40.183361 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:40.183179 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:40.183361 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:40.183267 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:40.183462 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:40.183371 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:41.186556 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:41.186359 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:41.186556 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:41.186467 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:41.668269 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:41.668223 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:41.668460 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:41.668438 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:41.668524 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:41.668514 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret podName:5e3e1d99-1fd1-4f01-980b-146f14cb0b43 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:49.668497593 +0000 UTC m=+17.112540763 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret") pod "global-pull-secret-syncer-cbl98" (UID: "5e3e1d99-1fd1-4f01-980b-146f14cb0b43") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:41.769538 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:41.769493 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:41.769726 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:41.769683 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:41.769864 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:41.769783 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs podName:9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:49.769760001 +0000 UTC m=+17.213803161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs") pod "network-metrics-daemon-b4bc4" (UID: "9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:41.870492 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:41.870453 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pghr\" (UniqueName: \"kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr\") pod \"network-check-target-w4grp\" (UID: \"ba088306-b1bd-4a02-9ee2-e547cc187b9f\") " pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:41.870668 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:41.870641 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:41.870668 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:41.870667 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:41.870793 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:41.870681 2574 projected.go:196] Error preparing data for projected volume kube-api-access-7pghr for pod openshift-network-diagnostics/network-check-target-w4grp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:41.870793 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:41.870761 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr podName:ba088306-b1bd-4a02-9ee2-e547cc187b9f nodeName:}" failed. No retries permitted until 2026-07-07 12:42:49.870741066 +0000 UTC m=+17.314784241 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pghr" (UniqueName: "kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr") pod "network-check-target-w4grp" (UID: "ba088306-b1bd-4a02-9ee2-e547cc187b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:42.183206 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:42.183159 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:42.183206 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:42.183192 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:42.183451 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:42.183287 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:42.183451 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:42.183436 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:43.184466 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:43.183970 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:43.184466 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:43.184126 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:44.183371 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:44.182947 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:44.183371 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:44.182994 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:44.183371 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:44.183076 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:44.183371 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:44.183229 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:45.183308 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:45.183268 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:45.183788 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:45.183400 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:46.183110 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:46.183078 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:46.183230 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:46.183197 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:46.183283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:46.183262 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:46.183392 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:46.183367 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:47.183448 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:47.183383 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:47.183915 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:47.183520 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:48.183442 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:48.183409 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:48.183612 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:48.183410 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:48.183612 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:48.183544 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:48.183612 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:48.183596 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:49.183275 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:49.183077 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:49.183443 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:49.183398 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:49.730345 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:49.730290 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:49.730750 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:49.730451 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:49.730750 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:49.730528 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret podName:5e3e1d99-1fd1-4f01-980b-146f14cb0b43 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:05.730506577 +0000 UTC m=+33.174549745 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret") pod "global-pull-secret-syncer-cbl98" (UID: "5e3e1d99-1fd1-4f01-980b-146f14cb0b43") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:49.831323 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:49.831287 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:49.831493 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:49.831435 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:49.831549 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:49.831508 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs podName:9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:05.831488351 +0000 UTC m=+33.275531507 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs") pod "network-metrics-daemon-b4bc4" (UID: "9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:49.932440 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:49.932407 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pghr\" (UniqueName: \"kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr\") pod \"network-check-target-w4grp\" (UID: \"ba088306-b1bd-4a02-9ee2-e547cc187b9f\") " pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:49.932625 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:49.932593 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:49.932625 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:49.932618 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:49.932764 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:49.932634 2574 projected.go:196] Error preparing data for projected volume kube-api-access-7pghr for pod openshift-network-diagnostics/network-check-target-w4grp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:49.932764 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:49.932703 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr podName:ba088306-b1bd-4a02-9ee2-e547cc187b9f nodeName:}" failed. No retries permitted until 2026-07-07 12:43:05.932676954 +0000 UTC m=+33.376720112 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pghr" (UniqueName: "kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr") pod "network-check-target-w4grp" (UID: "ba088306-b1bd-4a02-9ee2-e547cc187b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:50.183471 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:50.183391 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:50.183471 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:50.183415 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:50.183636 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:50.183525 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:50.183669 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:50.183637 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:51.183046 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:51.183009 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:51.183410 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:51.183141 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:52.183476 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:52.183440 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:52.183945 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:52.183510 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:52.183945 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:52.183621 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:52.183945 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:52.183707 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:53.183972 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:53.183939 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:53.184420 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:53.184048 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:54.182792 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:54.182760 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:54.182949 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:54.182762 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:54.182949 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:54.182866 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:54.183019 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:54.182954 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:55.185562 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.185385 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:55.186199 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:55.185648 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:55.319433 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.319196 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" event={"ID":"5ff2cc97-4b16-4c08-a180-4b8341bca5e9","Type":"ContainerStarted","Data":"dd8bba5ae1ef315cb922dec863faa5ef3708a3e2017470658cfb827b5a328738"} Jul 07 12:42:55.325724 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.325648 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" event={"ID":"60f064c6-f290-4194-8c69-9f0c1712498e","Type":"ContainerStarted","Data":"d54abd3d468139b4267dbb0d7c1c479cc80fca17e3ce1ad0223d76a0cff5b7b8"} Jul 07 12:42:55.325847 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.325751 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" event={"ID":"60f064c6-f290-4194-8c69-9f0c1712498e","Type":"ContainerStarted","Data":"b23ae7c678ad97032608588d3619576510f3ed00cc5c90767308947359c5a56c"} Jul 07 12:42:55.325847 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.325790 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" event={"ID":"60f064c6-f290-4194-8c69-9f0c1712498e","Type":"ContainerStarted","Data":"4d10ac2a94f3764034f01067feb7e741a959756b5dcf24fdeefc36d3a36594c6"} Jul 07 12:42:55.325847 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.325820 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" event={"ID":"60f064c6-f290-4194-8c69-9f0c1712498e","Type":"ContainerStarted","Data":"49f8949e950db1a69d8a26baf8d7043cbec2a61688d2e6c897b1518a291938c6"} Jul 07 12:42:55.326006 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.325851 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" event={"ID":"60f064c6-f290-4194-8c69-9f0c1712498e","Type":"ContainerStarted","Data":"dc533f713d0eb9d0abb3ed92975f8e49f19a50fdb50ba8cea39ba1c2ac9df0fe"} Jul 07 12:42:55.326006 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.325918 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" event={"ID":"60f064c6-f290-4194-8c69-9f0c1712498e","Type":"ContainerStarted","Data":"12f944f6caa445099e113e552ec19b215345ec72979af95ab887ef9359a0eed2"} Jul 07 12:42:55.328077 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.328051 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" event={"ID":"10f829cd-c046-4df6-a5d6-ceff68b1656a","Type":"ContainerStarted","Data":"2736a7771fc2a62b12bcee3d3249234c9649198b492fff7a122ad433e76fa448"} Jul 07 12:42:55.333317 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.333290 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2dtxn" event={"ID":"cc610f8c-e866-4ff6-8a27-172ddb224a0c","Type":"ContainerStarted","Data":"43040ef829c2aed6d8e469985863e0b9070263925c501742363f2463cc08a74f"} Jul 07 12:42:55.335182 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.335151 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-212.ec2.internal" event={"ID":"37ae1767e02e91e0c33f5647bfe3b14c","Type":"ContainerStarted","Data":"66548626587dc0db6bdaec5f210168e5547437e92b6ea0bd866dc19a10753086"} Jul 07 12:42:55.336734 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.336669 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xj9qm" podStartSLOduration=2.288628194 podStartE2EDuration="22.336650797s" totalImagesPullingTime="20.048022603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:55.336407504 +0000 UTC m=+22.780450700" watchObservedRunningTime="2026-07-07 12:42:55.336650797 +0000 UTC m=+22.780693976" Jul 07 12:42:55.337450 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.337424 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fbpcz" event={"ID":"87df0b3d-276d-4158-942c-4bdb9d08732d","Type":"ContainerStarted","Data":"e70176a31ada1d8b5d8eba6007abb8a9e16a4e51b37675202ad62b7ca98b6fe7"} Jul 07 12:42:55.352763 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.352723 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-212.ec2.internal" podStartSLOduration=22.352708843 podStartE2EDuration="22.352708843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:55.352505361 +0000 UTC m=+22.796548570" watchObservedRunningTime="2026-07-07 12:42:55.352708843 +0000 UTC m=+22.796752016" Jul 07 12:42:55.367188 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.367150 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2dtxn" podStartSLOduration=2.294728701 podStartE2EDuration="22.367136457s" totalImagesPullingTime="20.072407756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:55.366860511 +0000 UTC m=+22.810903709" watchObservedRunningTime="2026-07-07 12:42:55.367136457 +0000 UTC m=+22.811179637" Jul 07 12:42:55.511130 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.511051 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:55.568084 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.568054 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:55.675890 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.675839 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:55.738621 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.738582 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:55.800648 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.800583 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:55.901832 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:55.901806 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:56.002634 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.002603 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:56.183544 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.183440 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:56.183544 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.183486 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:56.183777 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:56.183585 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:56.183777 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:56.183672 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:56.209179 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.209150 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:56.270052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.270015 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:56.298130 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.298106 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:56.298658 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.298644 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:56.314496 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.314452 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fbpcz" podStartSLOduration=3.320091327 podStartE2EDuration="23.314434526s" totalImagesPullingTime="19.994343199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:55.384424774 +0000 UTC m=+22.828467972" watchObservedRunningTime="2026-07-07 12:42:56.314434526 +0000 UTC m=+23.758477704" Jul 07 12:42:56.343640 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.343607 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5pzkl" event={"ID":"be21f262-be06-421e-bb1e-f74edd4fffef","Type":"ContainerStarted","Data":"bf93b5e5ef600a529a62be9908d888eab4f5036c57b95c51b1d73c2ffc34432e"} Jul 07 12:42:56.345466 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.345378 2574 generic.go:356] "Generic (PLEG): container finished" podID="10f829cd-c046-4df6-a5d6-ceff68b1656a" containerID="2736a7771fc2a62b12bcee3d3249234c9649198b492fff7a122ad433e76fa448" exitCode=0 Jul 07 12:42:56.345466 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.345407 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" event={"ID":"10f829cd-c046-4df6-a5d6-ceff68b1656a","Type":"ContainerDied","Data":"2736a7771fc2a62b12bcee3d3249234c9649198b492fff7a122ad433e76fa448"} Jul 07 12:42:56.347009 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.346988 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" event={"ID":"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02","Type":"ContainerStarted","Data":"9f8304254f79da2ce21c269f58a2ac0c8e81246057d8d837d8b95d49c96b690c"} Jul 07 12:42:56.348459 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.348440 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qrr6k" event={"ID":"3a67391a-c8bc-4eaf-a00a-c728f059574c","Type":"ContainerStarted","Data":"fe8c9074f1d0863d40724d7c0ec0ad3a5dc2fbb33357d2908d151c8a28390849"} Jul 07 12:42:56.349790 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.349770 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6f6bh" event={"ID":"ae03df12-73df-481b-b330-c624c016e8cb","Type":"ContainerStarted","Data":"6855eb58b8034a158f3d79f502101c98c6410970e6c466a0fa448780b3dd6802"} Jul 07 12:42:56.351166 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.351148 2574 generic.go:356] "Generic (PLEG): container finished" podID="e763d3c31083b98290518ed2246097cf" containerID="0c86a2d0cd542fa5aba204cc018c64692b0ec11cbf7c8618d08a7a055e104882" exitCode=0 Jul 07 12:42:56.351251 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.351193 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" event={"ID":"e763d3c31083b98290518ed2246097cf","Type":"ContainerDied","Data":"0c86a2d0cd542fa5aba204cc018c64692b0ec11cbf7c8618d08a7a055e104882"} Jul 07 12:42:56.358114 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.358080 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5pzkl" podStartSLOduration=4.101035135 podStartE2EDuration="23.358068684s" totalImagesPullingTime="19.257033549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:56.357866765 +0000 UTC m=+23.801909943" watchObservedRunningTime="2026-07-07 12:42:56.358068684 +0000 UTC m=+23.802111863" Jul 07 12:42:56.416537 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.416466 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qrr6k" podStartSLOduration=11.795009664 podStartE2EDuration="23.416443631s" totalImagesPullingTime="11.621433967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:56.413952287 +0000 UTC m=+23.857995468" watchObservedRunningTime="2026-07-07 12:42:56.416443631 +0000 UTC m=+23.860486814" Jul 07 12:42:56.431008 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.430962 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6f6bh" podStartSLOduration=3.431882363 podStartE2EDuration="23.430946585s" totalImagesPullingTime="19.999064222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:56.430421317 +0000 UTC m=+23.874464516" watchObservedRunningTime="2026-07-07 12:42:56.430946585 +0000 UTC m=+23.874989766" Jul 07 12:42:56.459424 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.459396 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:56.559997 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.559962 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:56.658311 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.658277 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:56.759992 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.759931 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:56.783979 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.783955 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:56.784557 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.784533 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2dtxn" Jul 07 12:42:56.859388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.859215 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:56.925902 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.925875 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 12:42:56.960161 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:56.960127 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:57.062492 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:57.062336 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T12:42:56.925897541Z","UUID":"003da690-b62b-4e1e-841e-b55af7ebf745","Handler":null,"Name":"","Endpoint":""} Jul 07 12:42:57.064181 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:57.064162 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 12:42:57.064306 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:57.064205 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 12:42:57.121470 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:57.121429 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:57.186399 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:57.186374 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:57.186568 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:57.186463 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:57.216194 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:57.216169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:57.355863 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:57.355830 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" event={"ID":"e763d3c31083b98290518ed2246097cf","Type":"ContainerStarted","Data":"c2e034aef7535c66ba1dc791192be61230518e0ceb20563382beaa6f60caae6e"} Jul 07 12:42:57.359407 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:57.359375 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" event={"ID":"60f064c6-f290-4194-8c69-9f0c1712498e","Type":"ContainerStarted","Data":"d899e5486cbde95fb846f50ebdb8b1bc685bfedd5967b30d30c4b65e876fb6d3"} Jul 07 12:42:57.361406 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:57.361381 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" event={"ID":"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02","Type":"ContainerStarted","Data":"de2aca95dc08b82dadd1f81c29fed30c507dda72f6e4bc8c8ee2df38c328050c"} Jul 07 12:42:57.376461 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:57.376411 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-212.ec2.internal" podStartSLOduration=24.376394683 podStartE2EDuration="24.376394683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:57.375659159 +0000 UTC m=+24.819702338" watchObservedRunningTime="2026-07-07 12:42:57.376394683 +0000 UTC m=+24.820437861" Jul 07 12:42:58.183335 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:58.183312 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:42:58.183484 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:58.183418 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:42:58.183550 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:58.183477 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:42:58.183608 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:58.183589 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:42:58.218247 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:58.218226 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:58.281615 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:58.281583 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:58.351684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:58.351612 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6f6bh_ae03df12-73df-481b-b330-c624c016e8cb/dns-node-resolver/0.log" Jul 07 12:42:58.365825 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:58.365793 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" event={"ID":"2b81ef9a-ef17-4f72-a92a-d81a99fc3b02","Type":"ContainerStarted","Data":"483e6fb293a01d46d3744184148d49724b36c5ad3f14df504b82f64d0665d5f5"} Jul 07 12:42:58.386832 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:58.386781 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6vqk" podStartSLOduration=2.79088415 podStartE2EDuration="25.386767787s" totalImagesPullingTime="22.595883637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:58.386595335 +0000 UTC m=+25.830638530" watchObservedRunningTime="2026-07-07 12:42:58.386767787 +0000 UTC m=+25.830810966" Jul 07 12:42:59.090126 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:59.090096 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:59.183605 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:59.183564 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:42:59.183849 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:42:59.183713 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:42:59.191601 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:59.191572 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:42:59.552608 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:42:59.552375 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qrr6k_3a67391a-c8bc-4eaf-a00a-c728f059574c/node-ca/0.log" Jul 07 12:43:00.183664 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:00.183633 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:00.183840 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:00.183802 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:00.183890 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:00.183859 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:00.183994 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:00.183972 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:00.372948 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:00.372916 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" event={"ID":"60f064c6-f290-4194-8c69-9f0c1712498e","Type":"ContainerStarted","Data":"a30e9ee9f7f174beb98339fa5aec188bbb9a496ba7968a1daaff4bbb99cf2087"} Jul 07 12:43:00.373287 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:00.373249 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:43:00.373381 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:00.373322 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:43:00.389004 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:00.388978 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:43:00.403181 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:00.403133 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" podStartSLOduration=7.387438104 podStartE2EDuration="27.40311751s" totalImagesPullingTime="20.015679406s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:00.402737455 +0000 UTC m=+27.846780634" watchObservedRunningTime="2026-07-07 12:43:00.40311751 +0000 UTC m=+27.847160689" Jul 07 12:43:01.183620 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:01.183532 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:01.184406 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:01.183649 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:01.376123 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:01.376089 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:43:01.392318 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:01.392286 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:43:02.133381 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:02.133346 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:02.182927 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:02.182895 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:02.183080 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:02.182941 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:02.183080 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:02.183041 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:02.183168 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:02.183154 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:02.688989 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:02.688954 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:02.812316 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:02.812281 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:02.878703 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:02.878654 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:02.998893 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:02.998832 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:03.099852 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:03.099814 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:03.184218 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:03.184191 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:03.184331 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:03.184284 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:03.382375 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:03.382337 2574 generic.go:356] "Generic (PLEG): container finished" podID="10f829cd-c046-4df6-a5d6-ceff68b1656a" containerID="b38dfd06e03cdb783d5031018555578670cda23b7673a5e279ff56550531d166" exitCode=0 Jul 07 12:43:03.382520 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:03.382420 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" event={"ID":"10f829cd-c046-4df6-a5d6-ceff68b1656a","Type":"ContainerDied","Data":"b38dfd06e03cdb783d5031018555578670cda23b7673a5e279ff56550531d166"} Jul 07 12:43:03.765013 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:03.764844 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:04.183153 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:04.183055 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:04.183153 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:04.183089 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:04.183321 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:04.183175 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:04.183321 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:04.183261 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:04.366935 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:04.366907 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:04.558199 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:04.558172 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:04.973722 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:04.973676 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:05.149630 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:05.149600 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:05.183333 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:05.183310 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:05.183451 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:05.183430 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:05.350298 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:05.350226 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:05.387909 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:05.387876 2574 generic.go:356] "Generic (PLEG): container finished" podID="10f829cd-c046-4df6-a5d6-ceff68b1656a" containerID="3a176ff009ebe467614cda2d6565ad874f67e3069fd31ffdac2fc1479061da62" exitCode=0 Jul 07 12:43:05.388056 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:05.387925 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" event={"ID":"10f829cd-c046-4df6-a5d6-ceff68b1656a","Type":"ContainerDied","Data":"3a176ff009ebe467614cda2d6565ad874f67e3069fd31ffdac2fc1479061da62"} Jul 07 12:43:05.750522 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:05.750489 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:05.750708 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:05.750617 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:43:05.750708 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:05.750677 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret podName:5e3e1d99-1fd1-4f01-980b-146f14cb0b43 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:37.75066372 +0000 UTC m=+65.194706877 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret") pod "global-pull-secret-syncer-cbl98" (UID: "5e3e1d99-1fd1-4f01-980b-146f14cb0b43") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:43:05.850936 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:05.850904 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:05.851053 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:05.851022 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:43:05.851101 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:05.851072 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs podName:9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:37.851058135 +0000 UTC m=+65.295101292 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs") pod "network-metrics-daemon-b4bc4" (UID: "9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:43:05.952050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:05.952024 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pghr\" (UniqueName: \"kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr\") pod \"network-check-target-w4grp\" (UID: \"ba088306-b1bd-4a02-9ee2-e547cc187b9f\") " pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:05.952171 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:05.952152 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:43:05.952171 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:05.952165 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:43:05.952245 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:05.952175 2574 projected.go:196] Error preparing data for projected volume kube-api-access-7pghr for pod openshift-network-diagnostics/network-check-target-w4grp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:43:05.952245 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:05.952217 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr podName:ba088306-b1bd-4a02-9ee2-e547cc187b9f nodeName:}" failed. No retries permitted until 2026-07-07 12:43:37.952205548 +0000 UTC m=+65.396248706 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pghr" (UniqueName: "kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr") pod "network-check-target-w4grp" (UID: "ba088306-b1bd-4a02-9ee2-e547cc187b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:43:06.183281 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:06.183208 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:06.183281 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:06.183238 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:06.183634 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:06.183300 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:06.183634 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:06.183422 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:06.263743 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:06.263718 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:06.364543 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:06.364516 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:06.553997 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:06.553969 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:06.962585 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:06.962552 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:07.157192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:07.157169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:07.182865 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:07.182840 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:07.182960 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:07.182938 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:07.258601 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:07.258532 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:07.394270 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:07.394235 2574 generic.go:356] "Generic (PLEG): container finished" podID="10f829cd-c046-4df6-a5d6-ceff68b1656a" containerID="698630077f5d76ac57081498468b6e68cd9c2b3f3567c1d852de2a2511e73980" exitCode=0 Jul 07 12:43:07.394428 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:07.394278 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" event={"ID":"10f829cd-c046-4df6-a5d6-ceff68b1656a","Type":"ContainerDied","Data":"698630077f5d76ac57081498468b6e68cd9c2b3f3567c1d852de2a2511e73980"} Jul 07 12:43:07.799572 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:07.799542 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:08.183535 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:08.183461 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:08.183673 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:08.183461 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:08.183673 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:08.183577 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:08.183673 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:08.183641 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:08.366951 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:08.366919 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:08.558541 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:08.558504 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:08.757631 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:08.757598 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:08.816336 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:08.816263 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:08.889190 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:08.889160 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:08.967094 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:08.967058 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:09.155128 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:09.155053 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:09.183252 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:09.183217 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:09.183406 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:09.183345 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:09.953634 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:09.953603 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:10.157250 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:10.157214 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:10.183343 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:10.183315 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:10.183507 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:10.183441 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:10.183507 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:10.183500 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:10.183623 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:10.183597 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:10.452703 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:10.452663 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:10.554350 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:10.554292 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:10.754579 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:10.754505 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:10.953940 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:10.953889 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:11.183579 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:11.183492 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:11.183761 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:11.183636 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:11.964000 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:11.963968 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:12.183248 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:12.183219 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:12.183396 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:12.183254 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:12.183396 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:12.183336 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:12.183468 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:12.183446 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:12.364763 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:12.364740 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:12.560959 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:12.560921 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:12.765949 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:12.765916 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:13.159989 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:13.159898 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:13.184207 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:13.184168 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:13.184343 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:13.184261 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:13.357509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:13.357467 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:13.413107 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:13.413022 2574 generic.go:356] "Generic (PLEG): container finished" podID="10f829cd-c046-4df6-a5d6-ceff68b1656a" containerID="89be521e108c2bd30ffd826b1195877639fc5c2b8d421629ca7d38f9504c9633" exitCode=0 Jul 07 12:43:13.413107 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:13.413054 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" event={"ID":"10f829cd-c046-4df6-a5d6-ceff68b1656a","Type":"ContainerDied","Data":"89be521e108c2bd30ffd826b1195877639fc5c2b8d421629ca7d38f9504c9633"} Jul 07 12:43:14.183408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:14.183382 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:14.183792 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:14.183476 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:14.183792 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:14.183532 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:14.183792 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:14.183608 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:14.417833 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:14.417799 2574 generic.go:356] "Generic (PLEG): container finished" podID="10f829cd-c046-4df6-a5d6-ceff68b1656a" containerID="49f7a38d911701a278201e2ef175a236d64caf8390d23f62720b6dd3c60762d4" exitCode=0 Jul 07 12:43:14.417981 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:14.417849 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" event={"ID":"10f829cd-c046-4df6-a5d6-ceff68b1656a","Type":"ContainerDied","Data":"49f7a38d911701a278201e2ef175a236d64caf8390d23f62720b6dd3c60762d4"} Jul 07 12:43:15.185984 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:15.185958 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:15.186368 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:15.186046 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:15.423289 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:15.423254 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" event={"ID":"10f829cd-c046-4df6-a5d6-ceff68b1656a","Type":"ContainerStarted","Data":"4a158ad71521b186eae68e3fac86d27faa1e45d6b8041fc63680d1bfa22f3dbe"} Jul 07 12:43:15.446288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:15.446241 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hgkrq" podStartSLOduration=8.068403055 podStartE2EDuration="42.446226569s" totalImagesPullingTime="34.377823514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:15.445882084 +0000 UTC m=+42.889925285" watchObservedRunningTime="2026-07-07 12:43:15.446226569 +0000 UTC m=+42.890269747" Jul 07 12:43:16.183447 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:16.183288 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:16.183651 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:16.183611 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:16.186097 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:16.183910 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:16.186097 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:16.184278 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:17.185700 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:17.185659 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:17.185855 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:17.185763 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:18.182859 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:18.182822 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:18.183327 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:18.182832 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:18.183327 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:18.182927 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:18.183327 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:18.183008 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:18.977671 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:18.977637 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:19.079167 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:19.079137 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:19.179288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:19.179255 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:19.186051 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:19.186031 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:19.186398 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:19.186128 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:19.290451 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:19.290376 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:19.446795 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:19.446765 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:19.505572 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:19.505543 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:19.653034 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:19.652958 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:19.731875 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:19.731852 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:19.886633 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:19.886601 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:19.946914 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:19.946880 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:20.052272 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:20.052241 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:20.183582 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:20.183545 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:20.183740 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:20.183545 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:20.183740 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:20.183658 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:20.183831 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:20.183745 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:20.689002 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:20.688968 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:20.898223 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:20.898192 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:21.007874 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:21.007804 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:21.103492 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:21.103454 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:21.186267 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:21.186244 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:21.186425 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:21.186335 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:21.205597 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:21.205568 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:21.303405 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:21.303325 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:21.477061 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:21.477033 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:21.577557 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:21.577468 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:22.183023 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.182998 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:22.183408 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:22.183104 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:22.183408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.183161 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:22.183408 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:22.183297 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:22.626629 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.626458 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:22.634006 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.633967 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w4grp"] Jul 07 12:43:22.634174 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.634082 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:22.634240 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:22.634185 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:22.634765 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.634360 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b4bc4"] Jul 07 12:43:22.634765 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.634470 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:22.634765 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:22.634575 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:22.635203 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.635114 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cbl98"] Jul 07 12:43:22.635203 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.635190 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:22.635346 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:22.635277 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:22.695480 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.695447 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:22.792746 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.792653 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:22.865835 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:22.865801 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:23.194375 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:23.194342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:23.817992 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:23.817962 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:23.893754 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:23.893664 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:24.182963 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:24.182884 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:24.182963 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:24.182912 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:24.183127 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:24.182996 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:24.183127 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:24.183002 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:24.183127 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:24.183062 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:24.183221 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:24.183139 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:24.185985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:24.185964 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:24.243902 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:24.243874 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:26.183224 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:26.183190 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:26.183681 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:26.183190 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:26.183681 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:26.183190 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:26.183681 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:26.183514 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w4grp" podUID="ba088306-b1bd-4a02-9ee2-e547cc187b9f" Jul 07 12:43:26.183681 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:26.183447 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b4bc4" podUID="9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709" Jul 07 12:43:26.183681 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:26.183317 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbl98" podUID="5e3e1d99-1fd1-4f01-980b-146f14cb0b43" Jul 07 12:43:27.366622 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.366538 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-212.ec2.internal" event="NodeReady" Jul 07 12:43:27.367130 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.366657 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 12:43:27.417088 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.417052 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vfdz6"] Jul 07 12:43:27.418977 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.418952 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.419587 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.419564 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qxgh8"] Jul 07 12:43:27.421270 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.421256 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qxgh8" Jul 07 12:43:27.421955 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.421936 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 12:43:27.422173 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.422159 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 12:43:27.422253 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.422162 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ncl7q\"" Jul 07 12:43:27.423905 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.423888 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 12:43:27.424002 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.423939 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 12:43:27.424002 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.423965 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zphhp\"" Jul 07 12:43:27.424186 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.424173 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 12:43:27.432099 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.432081 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vfdz6"] Jul 07 12:43:27.448434 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.448413 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qxgh8"] Jul 07 12:43:27.511520 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.511490 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16e2ac83-43c2-439f-994a-943d545f9aba-metrics-tls\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.511520 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.511524 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16e2ac83-43c2-439f-994a-943d545f9aba-tmp-dir\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.511765 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.511546 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/16e2ac83-43c2-439f-994a-943d545f9aba-config-volume\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.511765 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.511565 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6xkf\" (UniqueName: \"kubernetes.io/projected/70dd9ad2-6e2e-492e-801b-6b9965957746-kube-api-access-m6xkf\") pod \"ingress-canary-qxgh8\" (UID: \"70dd9ad2-6e2e-492e-801b-6b9965957746\") " pod="openshift-ingress-canary/ingress-canary-qxgh8" Jul 07 12:43:27.511765 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.511627 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70dd9ad2-6e2e-492e-801b-6b9965957746-cert\") pod \"ingress-canary-qxgh8\" (UID: \"70dd9ad2-6e2e-492e-801b-6b9965957746\") " pod="openshift-ingress-canary/ingress-canary-qxgh8" Jul 07 12:43:27.511765 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.511748 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frgx5\" (UniqueName: \"kubernetes.io/projected/16e2ac83-43c2-439f-994a-943d545f9aba-kube-api-access-frgx5\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.517027 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.517002 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xvjb2"] Jul 07 12:43:27.519261 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.519244 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.521842 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.521803 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 12:43:27.522037 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.522018 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 12:43:27.522133 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.522020 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 12:43:27.522133 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.522020 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jdzl4\"" Jul 07 12:43:27.522133 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.522106 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 12:43:27.529729 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.529709 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xvjb2"] Jul 07 12:43:27.580003 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.579948 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:27.612670 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.612638 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frgx5\" (UniqueName: \"kubernetes.io/projected/16e2ac83-43c2-439f-994a-943d545f9aba-kube-api-access-frgx5\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.612670 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.612679 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16e2ac83-43c2-439f-994a-943d545f9aba-metrics-tls\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.612901 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.612712 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5da8ea0-29bd-4abf-803d-1f76999d78a3-crio-socket\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.612901 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.612734 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5da8ea0-29bd-4abf-803d-1f76999d78a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.612901 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.612785 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5da8ea0-29bd-4abf-803d-1f76999d78a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.612901 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.612820 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl6lw\" (UniqueName: \"kubernetes.io/projected/f5da8ea0-29bd-4abf-803d-1f76999d78a3-kube-api-access-wl6lw\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.613085 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.612924 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16e2ac83-43c2-439f-994a-943d545f9aba-tmp-dir\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.613085 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.612946 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5da8ea0-29bd-4abf-803d-1f76999d78a3-data-volume\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.613085 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.612972 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/16e2ac83-43c2-439f-994a-943d545f9aba-config-volume\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.613085 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.612995 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6xkf\" (UniqueName: \"kubernetes.io/projected/70dd9ad2-6e2e-492e-801b-6b9965957746-kube-api-access-m6xkf\") pod \"ingress-canary-qxgh8\" (UID: \"70dd9ad2-6e2e-492e-801b-6b9965957746\") " pod="openshift-ingress-canary/ingress-canary-qxgh8" Jul 07 12:43:27.613085 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.613023 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70dd9ad2-6e2e-492e-801b-6b9965957746-cert\") pod \"ingress-canary-qxgh8\" (UID: \"70dd9ad2-6e2e-492e-801b-6b9965957746\") " pod="openshift-ingress-canary/ingress-canary-qxgh8" Jul 07 12:43:27.613321 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.613271 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16e2ac83-43c2-439f-994a-943d545f9aba-tmp-dir\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.613593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.613574 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/16e2ac83-43c2-439f-994a-943d545f9aba-config-volume\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.616618 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.616601 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16e2ac83-43c2-439f-994a-943d545f9aba-metrics-tls\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.616770 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.616728 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70dd9ad2-6e2e-492e-801b-6b9965957746-cert\") pod \"ingress-canary-qxgh8\" (UID: \"70dd9ad2-6e2e-492e-801b-6b9965957746\") " pod="openshift-ingress-canary/ingress-canary-qxgh8" Jul 07 12:43:27.621408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.621387 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frgx5\" (UniqueName: \"kubernetes.io/projected/16e2ac83-43c2-439f-994a-943d545f9aba-kube-api-access-frgx5\") pod \"dns-default-vfdz6\" (UID: \"16e2ac83-43c2-439f-994a-943d545f9aba\") " pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.621736 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.621709 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6xkf\" (UniqueName: \"kubernetes.io/projected/70dd9ad2-6e2e-492e-801b-6b9965957746-kube-api-access-m6xkf\") pod \"ingress-canary-qxgh8\" (UID: \"70dd9ad2-6e2e-492e-801b-6b9965957746\") " pod="openshift-ingress-canary/ingress-canary-qxgh8" Jul 07 12:43:27.674664 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.674628 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:27.714038 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.714000 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5da8ea0-29bd-4abf-803d-1f76999d78a3-crio-socket\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.714038 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.714036 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5da8ea0-29bd-4abf-803d-1f76999d78a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.714269 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.714055 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5da8ea0-29bd-4abf-803d-1f76999d78a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.714269 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.714080 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl6lw\" (UniqueName: \"kubernetes.io/projected/f5da8ea0-29bd-4abf-803d-1f76999d78a3-kube-api-access-wl6lw\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.714269 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.714112 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5da8ea0-29bd-4abf-803d-1f76999d78a3-data-volume\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.714423 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.714231 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5da8ea0-29bd-4abf-803d-1f76999d78a3-crio-socket\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.714475 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.714450 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5da8ea0-29bd-4abf-803d-1f76999d78a3-data-volume\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.714632 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.714613 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5da8ea0-29bd-4abf-803d-1f76999d78a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.716357 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.716337 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5da8ea0-29bd-4abf-803d-1f76999d78a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.724170 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.724151 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl6lw\" (UniqueName: \"kubernetes.io/projected/f5da8ea0-29bd-4abf-803d-1f76999d78a3-kube-api-access-wl6lw\") pod \"insights-runtime-extractor-xvjb2\" (UID: \"f5da8ea0-29bd-4abf-803d-1f76999d78a3\") " pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.729131 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.729111 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:27.734041 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.734025 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qxgh8" Jul 07 12:43:27.774310 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.774204 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:27.828167 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.828140 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xvjb2" Jul 07 12:43:27.847996 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.847967 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:27.877128 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.877059 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qxgh8"] Jul 07 12:43:27.881192 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:27.881167 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70dd9ad2_6e2e_492e_801b_6b9965957746.slice/crio-198f7114733599c29312550e7470837003adabe0c9ecc699aa954259a18d9377 WatchSource:0}: Error finding container 198f7114733599c29312550e7470837003adabe0c9ecc699aa954259a18d9377: Status 404 returned error can't find the container with id 198f7114733599c29312550e7470837003adabe0c9ecc699aa954259a18d9377 Jul 07 12:43:27.888572 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.888547 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vfdz6"] Jul 07 12:43:27.891767 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:27.891717 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e2ac83_43c2_439f_994a_943d545f9aba.slice/crio-1e4cb046fcf87e826d834f843f0bad4cb3de5f0cbf2936927b43aeabc4550263 WatchSource:0}: Error finding container 1e4cb046fcf87e826d834f843f0bad4cb3de5f0cbf2936927b43aeabc4550263: Status 404 returned error can't find the container with id 1e4cb046fcf87e826d834f843f0bad4cb3de5f0cbf2936927b43aeabc4550263 Jul 07 12:43:27.959627 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:27.959594 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xvjb2"] Jul 07 12:43:27.962157 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:27.962135 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5da8ea0_29bd_4abf_803d_1f76999d78a3.slice/crio-43296d0b274cfed11232fab1752a55ddd5f658bca5a9ba5d33645976bee7eda8 WatchSource:0}: Error finding container 43296d0b274cfed11232fab1752a55ddd5f658bca5a9ba5d33645976bee7eda8: Status 404 returned error can't find the container with id 43296d0b274cfed11232fab1752a55ddd5f658bca5a9ba5d33645976bee7eda8 Jul 07 12:43:28.092077 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.092030 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75869989b6-q6q58"] Jul 07 12:43:28.094854 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.094837 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.099290 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.099269 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8vd9j\"" Jul 07 12:43:28.099837 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.099821 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 12:43:28.102832 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.102814 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 12:43:28.104240 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.104224 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 12:43:28.104334 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.104315 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:28.105059 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.105044 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 12:43:28.105253 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.105237 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 12:43:28.105322 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.105281 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 12:43:28.105926 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.105910 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 12:43:28.111880 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.111862 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75869989b6-q6q58"] Jul 07 12:43:28.183615 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.183521 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:28.183615 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.183576 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:28.183865 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.183668 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:28.186384 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.186361 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 12:43:28.186518 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.186361 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 12:43:28.186518 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.186445 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 12:43:28.186518 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.186509 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 12:43:28.186674 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.186601 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-849mb\"" Jul 07 12:43:28.186674 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.186661 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:28.186793 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.186777 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gxjtk\"" Jul 07 12:43:28.217629 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.217607 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-oauth-config\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.217769 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.217633 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-oauth-serving-cert\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.217769 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.217650 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-config\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.217769 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.217668 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-service-ca\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.217914 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.217794 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-serving-cert\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.217914 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.217828 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47p6n\" (UniqueName: \"kubernetes.io/projected/ceaa562d-5f18-410c-b672-36a6384fcbb5-kube-api-access-47p6n\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.319153 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.319121 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-serving-cert\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.319329 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.319165 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47p6n\" (UniqueName: \"kubernetes.io/projected/ceaa562d-5f18-410c-b672-36a6384fcbb5-kube-api-access-47p6n\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.319495 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.319464 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-oauth-config\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.319616 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.319510 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-oauth-serving-cert\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.319616 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.319538 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-config\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.319616 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.319560 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-service-ca\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.320654 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.320385 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-service-ca\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.321583 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.320924 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-oauth-serving-cert\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.321583 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.321538 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-config\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.322136 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.322115 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-serving-cert\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.322435 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.322416 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-oauth-config\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.328203 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.328177 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47p6n\" (UniqueName: \"kubernetes.io/projected/ceaa562d-5f18-410c-b672-36a6384fcbb5-kube-api-access-47p6n\") pod \"console-75869989b6-q6q58\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.403861 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.403822 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:28.457494 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.457394 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vfdz6" event={"ID":"16e2ac83-43c2-439f-994a-943d545f9aba","Type":"ContainerStarted","Data":"1e4cb046fcf87e826d834f843f0bad4cb3de5f0cbf2936927b43aeabc4550263"} Jul 07 12:43:28.459815 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.459723 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xvjb2" event={"ID":"f5da8ea0-29bd-4abf-803d-1f76999d78a3","Type":"ContainerStarted","Data":"43296d0b274cfed11232fab1752a55ddd5f658bca5a9ba5d33645976bee7eda8"} Jul 07 12:43:28.463205 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.463174 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qxgh8" event={"ID":"70dd9ad2-6e2e-492e-801b-6b9965957746","Type":"ContainerStarted","Data":"198f7114733599c29312550e7470837003adabe0c9ecc699aa954259a18d9377"} Jul 07 12:43:28.578064 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:28.578029 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75869989b6-q6q58"] Jul 07 12:43:28.586766 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:28.586736 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceaa562d_5f18_410c_b672_36a6384fcbb5.slice/crio-cccc65f5e8f512567ab4e1e40095ad4f75aa98b90697dc7ed9dc62344406663d WatchSource:0}: Error finding container cccc65f5e8f512567ab4e1e40095ad4f75aa98b90697dc7ed9dc62344406663d: Status 404 returned error can't find the container with id cccc65f5e8f512567ab4e1e40095ad4f75aa98b90697dc7ed9dc62344406663d Jul 07 12:43:29.468526 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.468483 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75869989b6-q6q58" event={"ID":"ceaa562d-5f18-410c-b672-36a6384fcbb5","Type":"ContainerStarted","Data":"cccc65f5e8f512567ab4e1e40095ad4f75aa98b90697dc7ed9dc62344406663d"} Jul 07 12:43:29.830287 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.830128 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:29.885168 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.885117 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2"] Jul 07 12:43:29.887482 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.887456 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:29.891009 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.890989 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 12:43:29.891286 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.891267 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 12:43:29.893379 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.892442 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-b7v7d\"" Jul 07 12:43:29.893379 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.892537 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 12:43:29.893379 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.892442 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 12:43:29.893379 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.892736 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 12:43:29.902349 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.902324 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2"] Jul 07 12:43:29.929476 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.929439 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:29.936163 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.936136 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n4zm\" (UniqueName: \"kubernetes.io/projected/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-kube-api-access-4n4zm\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:29.936287 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.936183 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:29.936287 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.936216 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:29.936287 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:29.936272 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:30.013798 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.013730 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:30.037404 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.037371 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n4zm\" (UniqueName: \"kubernetes.io/projected/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-kube-api-access-4n4zm\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:30.037570 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.037423 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:30.037570 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.037453 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:30.037570 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.037488 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:30.038319 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.038295 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:30.040399 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.040366 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:30.041129 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.041107 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:30.046309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.046267 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n4zm\" (UniqueName: \"kubernetes.io/projected/ffacd4a8-e132-44cc-9bf9-a280a39a0daf-kube-api-access-4n4zm\") pod \"prometheus-operator-567f7b8b7c-7qjv2\" (UID: \"ffacd4a8-e132-44cc-9bf9-a280a39a0daf\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:30.201242 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.201211 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" Jul 07 12:43:30.286163 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.286123 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c7cc86965-hll8s"] Jul 07 12:43:30.290356 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.290327 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.292206 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.292177 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:30.301321 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.301298 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 12:43:30.301649 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.301406 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c7cc86965-hll8s"] Jul 07 12:43:30.339948 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.339917 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-oauth-serving-cert\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.340093 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.339962 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-trusted-ca-bundle\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.340093 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.340067 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-config\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.340185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.340141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-oauth-config\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.340231 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.340189 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-service-ca\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.340231 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.340221 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2qxw\" (UniqueName: \"kubernetes.io/projected/ed4e2b39-79c5-4fbd-addc-2277488c6160-kube-api-access-z2qxw\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.340306 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.340289 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-serving-cert\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.354339 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.354308 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:30.442217 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.441345 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-oauth-config\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.442217 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.441408 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-service-ca\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.442217 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.441433 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2qxw\" (UniqueName: \"kubernetes.io/projected/ed4e2b39-79c5-4fbd-addc-2277488c6160-kube-api-access-z2qxw\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.442217 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.441512 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-serving-cert\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.442217 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.441568 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-oauth-serving-cert\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.442217 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.441598 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-trusted-ca-bundle\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.442217 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.441634 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-config\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.442743 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.442260 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-service-ca\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.442743 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.442297 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-config\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.442743 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.442471 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-oauth-serving-cert\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.442743 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.442628 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-trusted-ca-bundle\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.444376 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.444347 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-oauth-config\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.444504 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.444458 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-serving-cert\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.452173 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.452107 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2qxw\" (UniqueName: \"kubernetes.io/projected/ed4e2b39-79c5-4fbd-addc-2277488c6160-kube-api-access-z2qxw\") pod \"console-7c7cc86965-hll8s\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:30.602485 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:30.602302 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:33.393971 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:33.393937 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kl9px" Jul 07 12:43:33.522969 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:33.522939 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:33.619437 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:33.619352 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:33.721152 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:33.721120 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:33.788594 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:33.788557 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:34.742026 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:34.742000 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2"] Jul 07 12:43:34.744966 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:34.744938 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffacd4a8_e132_44cc_9bf9_a280a39a0daf.slice/crio-bf276060e8dd3ea28ed6855a0db4d1edede236835159e23ac7c657fa1c9a32eb WatchSource:0}: Error finding container bf276060e8dd3ea28ed6855a0db4d1edede236835159e23ac7c657fa1c9a32eb: Status 404 returned error can't find the container with id bf276060e8dd3ea28ed6855a0db4d1edede236835159e23ac7c657fa1c9a32eb Jul 07 12:43:34.759817 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:34.759788 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c7cc86965-hll8s"] Jul 07 12:43:34.764069 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:34.764040 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded4e2b39_79c5_4fbd_addc_2277488c6160.slice/crio-6b72f45754e60df83bf4a0afb2ed2de421cc90b51cc6f857ad41168c3a45be40 WatchSource:0}: Error finding container 6b72f45754e60df83bf4a0afb2ed2de421cc90b51cc6f857ad41168c3a45be40: Status 404 returned error can't find the container with id 6b72f45754e60df83bf4a0afb2ed2de421cc90b51cc6f857ad41168c3a45be40 Jul 07 12:43:35.486908 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.486847 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" event={"ID":"ffacd4a8-e132-44cc-9bf9-a280a39a0daf","Type":"ContainerStarted","Data":"bf276060e8dd3ea28ed6855a0db4d1edede236835159e23ac7c657fa1c9a32eb"} Jul 07 12:43:35.489365 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.489338 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xvjb2" event={"ID":"f5da8ea0-29bd-4abf-803d-1f76999d78a3","Type":"ContainerStarted","Data":"ab8f539f10280c94c3850c5c64ac9186c5b019662ef2ac079ef944fa7dc0c963"} Jul 07 12:43:35.489481 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.489388 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xvjb2" event={"ID":"f5da8ea0-29bd-4abf-803d-1f76999d78a3","Type":"ContainerStarted","Data":"40323c6f2b5ce295e106820ea3e254ca515c7af49523ace99d251160766c3e8f"} Jul 07 12:43:35.491190 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.491160 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qxgh8" event={"ID":"70dd9ad2-6e2e-492e-801b-6b9965957746","Type":"ContainerStarted","Data":"ea78551fe02d051e745c2d3aefe9098f346e85898b381f711257f1931356591e"} Jul 07 12:43:35.493117 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.493095 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vfdz6" event={"ID":"16e2ac83-43c2-439f-994a-943d545f9aba","Type":"ContainerStarted","Data":"5ac3a0855291389985d3890742308b8800096036dfbdc05869eda495c9a377fc"} Jul 07 12:43:35.493223 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.493140 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vfdz6" event={"ID":"16e2ac83-43c2-439f-994a-943d545f9aba","Type":"ContainerStarted","Data":"19370385db835de2868aa487436cfbf5a1ec2906396ed23d0fef401168a3b989"} Jul 07 12:43:35.493223 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.493170 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:35.496538 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.496511 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75869989b6-q6q58" event={"ID":"ceaa562d-5f18-410c-b672-36a6384fcbb5","Type":"ContainerStarted","Data":"ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf"} Jul 07 12:43:35.498247 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.498224 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7cc86965-hll8s" event={"ID":"ed4e2b39-79c5-4fbd-addc-2277488c6160","Type":"ContainerStarted","Data":"470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683"} Jul 07 12:43:35.498344 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.498262 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7cc86965-hll8s" event={"ID":"ed4e2b39-79c5-4fbd-addc-2277488c6160","Type":"ContainerStarted","Data":"6b72f45754e60df83bf4a0afb2ed2de421cc90b51cc6f857ad41168c3a45be40"} Jul 07 12:43:35.508385 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.508340 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qxgh8" podStartSLOduration=1.797599286 podStartE2EDuration="8.508322666s" totalImagesPullingTime="6.71072338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:35.507626123 +0000 UTC m=+62.951669320" watchObservedRunningTime="2026-07-07 12:43:35.508322666 +0000 UTC m=+62.952365848" Jul 07 12:43:35.524989 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.524941 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75869989b6-q6q58" podStartSLOduration=1.5116676679999999 podStartE2EDuration="7.524925032s" totalImagesPullingTime="6.013257364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:35.524123734 +0000 UTC m=+62.968166934" watchObservedRunningTime="2026-07-07 12:43:35.524925032 +0000 UTC m=+62.968968213" Jul 07 12:43:35.540090 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.540048 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vfdz6" podStartSLOduration=1.8633602 podStartE2EDuration="8.540033693s" totalImagesPullingTime="6.676673493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:35.538878498 +0000 UTC m=+62.982921695" watchObservedRunningTime="2026-07-07 12:43:35.540033693 +0000 UTC m=+62.984076849" Jul 07 12:43:35.558424 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:35.558380 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c7cc86965-hll8s" podStartSLOduration=5.558364524 podStartE2EDuration="5.558364524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:35.557536606 +0000 UTC m=+63.001579787" watchObservedRunningTime="2026-07-07 12:43:35.558364524 +0000 UTC m=+63.002407704" Jul 07 12:43:36.000641 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:36.000599 2574 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/openshift/origin-insights-runtime-extractor:latest" Jul 07 12:43:36.000998 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:36.000747 2574 kuberuntime_manager.go:1459] "Unhandled Error" err="container extractor start failed in pod insights-runtime-extractor-xvjb2_openshift-insights(f5da8ea0-29bd-4abf-803d-1f76999d78a3): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 07 12:43:36.000998 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:36.000796 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extractor\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-insights/insights-runtime-extractor-xvjb2" podUID="f5da8ea0-29bd-4abf-803d-1f76999d78a3" Jul 07 12:43:36.039862 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:36.039836 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:36.139871 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:36.139789 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:36.201069 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:36.201037 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:36.505462 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:36.505421 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" event={"ID":"ffacd4a8-e132-44cc-9bf9-a280a39a0daf","Type":"ContainerStarted","Data":"e562a82e16c231ef45698803a01b8687b39b3cfba54b3303b92e45143bee38e9"} Jul 07 12:43:36.505646 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:36.505530 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" event={"ID":"ffacd4a8-e132-44cc-9bf9-a280a39a0daf","Type":"ContainerStarted","Data":"5ab9f5d9000531226814b36af46b8e850e1cff44782b81b7334e31edbd719303"} Jul 07 12:43:36.506246 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:36.506215 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extractor\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift/origin-insights-runtime-extractor:latest\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-insights/insights-runtime-extractor-xvjb2" podUID="f5da8ea0-29bd-4abf-803d-1f76999d78a3" Jul 07 12:43:36.536868 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:36.536820 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7qjv2" podStartSLOduration=6.426512099 podStartE2EDuration="7.536801316s" totalImagesPullingTime="1.110289217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:36.536324306 +0000 UTC m=+63.980367484" watchObservedRunningTime="2026-07-07 12:43:36.536801316 +0000 UTC m=+63.980844497" Jul 07 12:43:37.803946 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:37.803906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:37.807014 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:37.806980 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 12:43:37.816443 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:37.816416 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5e3e1d99-1fd1-4f01-980b-146f14cb0b43-original-pull-secret\") pod \"global-pull-secret-syncer-cbl98\" (UID: \"5e3e1d99-1fd1-4f01-980b-146f14cb0b43\") " pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:37.896946 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:37.896914 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:37.905082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:37.905049 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:37.907532 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:37.907516 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 12:43:37.917461 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:37.917437 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709-metrics-certs\") pod \"network-metrics-daemon-b4bc4\" (UID: \"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709\") " pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:37.997281 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:37.997251 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:38.006408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.006382 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pghr\" (UniqueName: \"kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr\") pod \"network-check-target-w4grp\" (UID: \"ba088306-b1bd-4a02-9ee2-e547cc187b9f\") " pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:38.009055 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.009040 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 12:43:38.019841 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.019822 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 12:43:38.030278 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.030255 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pghr\" (UniqueName: \"kubernetes.io/projected/ba088306-b1bd-4a02-9ee2-e547cc187b9f-kube-api-access-7pghr\") pod \"network-check-target-w4grp\" (UID: \"ba088306-b1bd-4a02-9ee2-e547cc187b9f\") " pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:38.096528 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.096456 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-849mb\"" Jul 07 12:43:38.100564 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.100549 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbl98" Jul 07 12:43:38.104434 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.104408 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:38.106566 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.106552 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gxjtk\"" Jul 07 12:43:38.115143 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.115119 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b4bc4" Jul 07 12:43:38.157264 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.157236 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:38.213702 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.213661 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b"] Jul 07 12:43:38.217196 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.217162 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.220181 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.220003 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7kqcp\"" Jul 07 12:43:38.220181 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.220032 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 12:43:38.220973 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.220952 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 12:43:38.230834 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.230810 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b"] Jul 07 12:43:38.240728 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.240675 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr"] Jul 07 12:43:38.240928 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.240819 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:38.243306 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.243285 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z6fdd"] Jul 07 12:43:38.244229 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.244163 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.248020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.247988 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 12:43:38.248660 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.248258 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 12:43:38.248660 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.248263 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 12:43:38.248660 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.248259 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.248660 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.248470 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nxkn8\"" Jul 07 12:43:38.251449 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.251222 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 12:43:38.251449 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.251233 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 12:43:38.251449 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.251247 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w4kcv\"" Jul 07 12:43:38.253115 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.252654 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 12:43:38.254730 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.254710 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w4grp"] Jul 07 12:43:38.255880 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.255861 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr"] Jul 07 12:43:38.258328 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:38.258304 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba088306_b1bd_4a02_9ee2_e547cc187b9f.slice/crio-6b271c0f88203e77932f8a271f62495b0ee9b0550ba450b2d6e0c28ec58a6d1e WatchSource:0}: Error finding container 6b271c0f88203e77932f8a271f62495b0ee9b0550ba450b2d6e0c28ec58a6d1e: Status 404 returned error can't find the container with id 6b271c0f88203e77932f8a271f62495b0ee9b0550ba450b2d6e0c28ec58a6d1e Jul 07 12:43:38.308679 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.308651 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.308818 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.308713 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7fnt\" (UniqueName: \"kubernetes.io/projected/ce62cb92-033c-4abc-a117-b3896d689f23-kube-api-access-z7fnt\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.308818 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.308761 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.308818 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.308779 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-sys\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.308818 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.308795 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-wtmp\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.308952 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.308819 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3644ab2c-4c37-4b02-b77c-bee617c5ed46-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.308952 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.308848 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-metrics-client-ca\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.308952 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.308894 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-textfile\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.308952 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.308918 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-tls\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.309072 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.308962 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3644ab2c-4c37-4b02-b77c-bee617c5ed46-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.309072 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.308988 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3644ab2c-4c37-4b02-b77c-bee617c5ed46-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.309072 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.309007 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg28p\" (UniqueName: \"kubernetes.io/projected/3644ab2c-4c37-4b02-b77c-bee617c5ed46-kube-api-access-tg28p\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.309072 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.309023 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.309072 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.309045 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ce62cb92-033c-4abc-a117-b3896d689f23-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.309212 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.309073 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.309212 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.309098 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.309212 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.309121 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce62cb92-033c-4abc-a117-b3896d689f23-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.309212 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.309148 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-root\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.309212 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.309165 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpb8h\" (UniqueName: \"kubernetes.io/projected/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-kube-api-access-jpb8h\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.351533 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.351472 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:38.404535 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.404511 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:38.404652 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.404563 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:38.409173 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409158 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:38.409399 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409383 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7fnt\" (UniqueName: \"kubernetes.io/projected/ce62cb92-033c-4abc-a117-b3896d689f23-kube-api-access-z7fnt\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.409469 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409421 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.409469 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409446 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-sys\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.409574 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409470 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-wtmp\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.409574 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409519 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3644ab2c-4c37-4b02-b77c-bee617c5ed46-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.409574 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409544 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-metrics-client-ca\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.409574 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:38.409523 2574 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 07 12:43:38.409574 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409569 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-sys\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.409836 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409584 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-textfile\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.409836 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409610 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-tls\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.409836 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:38.409625 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-tls podName:ce62cb92-033c-4abc-a117-b3896d689f23 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:38.90960414 +0000 UTC m=+66.353647300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-gr8cr" (UID: "ce62cb92-033c-4abc-a117-b3896d689f23") : secret "kube-state-metrics-tls" not found Jul 07 12:43:38.409836 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409643 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-wtmp\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.409836 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:38.409709 2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 12:43:38.409836 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409728 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3644ab2c-4c37-4b02-b77c-bee617c5ed46-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.409836 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:38.409767 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-tls podName:6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e nodeName:}" failed. No retries permitted until 2026-07-07 12:43:38.909750885 +0000 UTC m=+66.353794061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-tls") pod "node-exporter-z6fdd" (UID: "6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e") : secret "node-exporter-tls" not found Jul 07 12:43:38.409836 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409787 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3644ab2c-4c37-4b02-b77c-bee617c5ed46-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.409836 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409817 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg28p\" (UniqueName: \"kubernetes.io/projected/3644ab2c-4c37-4b02-b77c-bee617c5ed46-kube-api-access-tg28p\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.410277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409844 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.410277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409880 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ce62cb92-033c-4abc-a117-b3896d689f23-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.410277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.410277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409936 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.410277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409964 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce62cb92-033c-4abc-a117-b3896d689f23-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.410277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.409994 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-root\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.410277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.410022 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpb8h\" (UniqueName: \"kubernetes.io/projected/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-kube-api-access-jpb8h\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.410277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.410051 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-textfile\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.410277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.410069 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.410733 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.410432 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-root\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.410733 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.410473 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3644ab2c-4c37-4b02-b77c-bee617c5ed46-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.410733 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.410482 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-accelerators-collector-config\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.410881 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.410753 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ce62cb92-033c-4abc-a117-b3896d689f23-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.410881 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.410816 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.410881 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.410829 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-metrics-client-ca\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.411020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.410912 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce62cb92-033c-4abc-a117-b3896d689f23-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.412336 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.412301 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.412437 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.412419 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3644ab2c-4c37-4b02-b77c-bee617c5ed46-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.412705 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.412670 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.413149 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.413132 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3644ab2c-4c37-4b02-b77c-bee617c5ed46-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.417947 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.417915 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7fnt\" (UniqueName: \"kubernetes.io/projected/ce62cb92-033c-4abc-a117-b3896d689f23-kube-api-access-z7fnt\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.418266 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.418247 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg28p\" (UniqueName: \"kubernetes.io/projected/3644ab2c-4c37-4b02-b77c-bee617c5ed46-kube-api-access-tg28p\") pod \"openshift-state-metrics-54cf96dfcf-wnn8b\" (UID: \"3644ab2c-4c37-4b02-b77c-bee617c5ed46\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.418798 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.418781 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpb8h\" (UniqueName: \"kubernetes.io/projected/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-kube-api-access-jpb8h\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.472839 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.472806 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b4bc4"] Jul 07 12:43:38.475745 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:38.475713 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bbc4f72_0795_4ff1_b0ad_1dc68d5fb709.slice/crio-e373eb0ee2fdcf14a528ca7e3db406f4e20dc8044e8ed593dcd24bca120d8828 WatchSource:0}: Error finding container e373eb0ee2fdcf14a528ca7e3db406f4e20dc8044e8ed593dcd24bca120d8828: Status 404 returned error can't find the container with id e373eb0ee2fdcf14a528ca7e3db406f4e20dc8044e8ed593dcd24bca120d8828 Jul 07 12:43:38.475745 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.475737 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cbl98"] Jul 07 12:43:38.478813 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:38.478790 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e3e1d99_1fd1_4f01_980b_146f14cb0b43.slice/crio-1b08c91746ac54d0ca1d4c804416ff29faa7a5da9106fbeea421b97ad3f93bb0 WatchSource:0}: Error finding container 1b08c91746ac54d0ca1d4c804416ff29faa7a5da9106fbeea421b97ad3f93bb0: Status 404 returned error can't find the container with id 1b08c91746ac54d0ca1d4c804416ff29faa7a5da9106fbeea421b97ad3f93bb0 Jul 07 12:43:38.496135 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.496117 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:38.512533 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.512507 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b4bc4" event={"ID":"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709","Type":"ContainerStarted","Data":"e373eb0ee2fdcf14a528ca7e3db406f4e20dc8044e8ed593dcd24bca120d8828"} Jul 07 12:43:38.513571 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.513548 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w4grp" event={"ID":"ba088306-b1bd-4a02-9ee2-e547cc187b9f","Type":"ContainerStarted","Data":"6b271c0f88203e77932f8a271f62495b0ee9b0550ba450b2d6e0c28ec58a6d1e"} Jul 07 12:43:38.514598 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.514577 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cbl98" event={"ID":"5e3e1d99-1fd1-4f01-980b-146f14cb0b43","Type":"ContainerStarted","Data":"1b08c91746ac54d0ca1d4c804416ff29faa7a5da9106fbeea421b97ad3f93bb0"} Jul 07 12:43:38.518355 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.518335 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:43:38.530820 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.530798 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" Jul 07 12:43:38.655413 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.655393 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b"] Jul 07 12:43:38.657753 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:38.657726 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3644ab2c_4c37_4b02_b77c_bee617c5ed46.slice/crio-d07b0d3a8dc89329bf65f5ba870ca7cb8eb856660d1224ecfe918c87845054d3 WatchSource:0}: Error finding container d07b0d3a8dc89329bf65f5ba870ca7cb8eb856660d1224ecfe918c87845054d3: Status 404 returned error can't find the container with id d07b0d3a8dc89329bf65f5ba870ca7cb8eb856660d1224ecfe918c87845054d3 Jul 07 12:43:38.690822 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.690799 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:38.791924 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.791894 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:38.892520 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.892443 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:38.914619 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.914588 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:38.914796 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.914663 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-tls\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.917250 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.917230 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e-node-exporter-tls\") pod \"node-exporter-z6fdd\" (UID: \"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e\") " pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:38.917581 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:38.917562 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce62cb92-033c-4abc-a117-b3896d689f23-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gr8cr\" (UID: \"ce62cb92-033c-4abc-a117-b3896d689f23\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:39.163432 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.163165 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" Jul 07 12:43:39.176414 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.168878 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z6fdd" Jul 07 12:43:39.252190 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.252144 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:39.315391 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.313366 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:43:39.323215 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.323164 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.328527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.326304 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 12:43:39.328527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.326580 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 12:43:39.328527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.326790 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-njdct\"" Jul 07 12:43:39.328527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.327116 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 12:43:39.328527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.327292 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 12:43:39.328527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.327504 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 12:43:39.328527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.327701 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 12:43:39.328527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.327886 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 12:43:39.328527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.328153 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 12:43:39.328527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.328329 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 12:43:39.338549 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.338440 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:39.352190 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.351670 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:43:39.375938 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.375863 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr"] Jul 07 12:43:39.384705 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:39.384663 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce62cb92_033c_4abc_a117_b3896d689f23.slice/crio-0fe4a04fa2947886d770532e09a1eb93431140c43661a9a617d400842c463f39 WatchSource:0}: Error finding container 0fe4a04fa2947886d770532e09a1eb93431140c43661a9a617d400842c463f39: Status 404 returned error can't find the container with id 0fe4a04fa2947886d770532e09a1eb93431140c43661a9a617d400842c463f39 Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.432737 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.432794 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-config-volume\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.432825 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.432857 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.432888 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk9mm\" (UniqueName: \"kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-kube-api-access-fk9mm\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.432943 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-config-out\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.432968 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.432990 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.433029 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.433060 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.433086 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-web-config\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.433112 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.433330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.433134 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.461597 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.461562 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:39.523245 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.523167 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z6fdd" event={"ID":"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e","Type":"ContainerStarted","Data":"5e4a5a38231412a56d4684ec4150d39ed93681eaa7394bbdc71fad52134df4e6"} Jul 07 12:43:39.526276 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.526148 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" event={"ID":"3644ab2c-4c37-4b02-b77c-bee617c5ed46","Type":"ContainerStarted","Data":"c9f1bb266d68c2c672c42e19217b17449c84922c174598f8c1b5f44eb0aadf73"} Jul 07 12:43:39.526276 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.526210 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" event={"ID":"3644ab2c-4c37-4b02-b77c-bee617c5ed46","Type":"ContainerStarted","Data":"7b7aa39b8fa3c025877da46ea01a94bce625cf3c9a93717bde003ccfa0a2bb89"} Jul 07 12:43:39.526276 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.526240 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" event={"ID":"3644ab2c-4c37-4b02-b77c-bee617c5ed46","Type":"ContainerStarted","Data":"d07b0d3a8dc89329bf65f5ba870ca7cb8eb856660d1224ecfe918c87845054d3"} Jul 07 12:43:39.527710 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.527667 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" event={"ID":"ce62cb92-033c-4abc-a117-b3896d689f23","Type":"ContainerStarted","Data":"0fe4a04fa2947886d770532e09a1eb93431140c43661a9a617d400842c463f39"} Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534460 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534490 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-web-config\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534518 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534541 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534575 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534612 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-config-volume\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534633 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534664 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534714 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk9mm\" (UniqueName: \"kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-kube-api-access-fk9mm\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534778 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-config-out\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534804 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.534828 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.535684 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.535227 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.538076 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.536647 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.538076 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.536915 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.538076 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:39.537477 2574 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 07 12:43:39.538076 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:39.537557 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-main-tls podName:23deede0-66e8-4510-87b3-e529f030053f nodeName:}" failed. No retries permitted until 2026-07-07 12:43:40.037537257 +0000 UTC m=+67.481580421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "23deede0-66e8-4510-87b3-e529f030053f") : secret "alertmanager-main-tls" not found Jul 07 12:43:39.543908 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.543881 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.544363 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.544326 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-config-out\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.544889 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.544865 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.545718 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.545432 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-web-config\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.545718 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.545464 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.545718 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.545580 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk9mm\" (UniqueName: \"kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-kube-api-access-fk9mm\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.546136 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.545946 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-config-volume\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.546136 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.546100 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:39.547043 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:39.547002 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:40.039635 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.039219 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:40.042530 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.042499 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:40.157305 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.157269 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:40.253640 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.253601 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:43:40.258206 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.258054 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:40.306446 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.306368 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6686974f76-rmcwm"] Jul 07 12:43:40.309926 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.309900 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.314204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.314157 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-tn8wb\"" Jul 07 12:43:40.315732 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.315477 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 12:43:40.315732 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.315472 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 12:43:40.315732 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.315639 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6ju87jb8h5sav\"" Jul 07 12:43:40.315976 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.315961 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 12:43:40.316932 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.316913 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 12:43:40.320088 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.319925 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 12:43:40.335487 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.335407 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6686974f76-rmcwm"] Jul 07 12:43:40.445096 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.445046 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-grpc-tls\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.445096 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.445093 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4l44\" (UniqueName: \"kubernetes.io/projected/b6591db7-8228-485c-aa88-13ebba70998c-kube-api-access-m4l44\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.445321 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.445125 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-tls\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.445321 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.445198 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.445321 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.445264 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.445321 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.445295 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.445554 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.445417 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.445554 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.445477 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6591db7-8228-485c-aa88-13ebba70998c-metrics-client-ca\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.546672 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.546635 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6591db7-8228-485c-aa88-13ebba70998c-metrics-client-ca\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.546861 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.546748 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-grpc-tls\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.546861 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.546782 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4l44\" (UniqueName: \"kubernetes.io/projected/b6591db7-8228-485c-aa88-13ebba70998c-kube-api-access-m4l44\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.546861 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.546817 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-tls\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.546861 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.546847 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.547056 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.546882 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.547056 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.546925 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.547056 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.547027 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.547568 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.547422 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6591db7-8228-485c-aa88-13ebba70998c-metrics-client-ca\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.551710 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.551663 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.551844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.551663 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.552048 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.552002 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.552624 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.552603 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-grpc-tls\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.552736 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.552638 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.555132 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.555089 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b6591db7-8228-485c-aa88-13ebba70998c-secret-thanos-querier-tls\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.558287 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.558239 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4l44\" (UniqueName: \"kubernetes.io/projected/b6591db7-8228-485c-aa88-13ebba70998c-kube-api-access-m4l44\") pod \"thanos-querier-6686974f76-rmcwm\" (UID: \"b6591db7-8228-485c-aa88-13ebba70998c\") " pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.603364 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.603301 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:40.603459 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.603376 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:40.610190 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.609995 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:40.623224 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.623201 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:40.777619 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.776202 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:43:40.834668 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:40.834641 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6686974f76-rmcwm"] Jul 07 12:43:41.211202 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.211169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:41.218737 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:41.218656 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6591db7_8228_485c_aa88_13ebba70998c.slice/crio-20d082629019791aab0428647451d53f41033f6b384d11789e9bf956e8658203 WatchSource:0}: Error finding container 20d082629019791aab0428647451d53f41033f6b384d11789e9bf956e8658203: Status 404 returned error can't find the container with id 20d082629019791aab0428647451d53f41033f6b384d11789e9bf956e8658203 Jul 07 12:43:41.311875 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.311847 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:41.503421 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.503304 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:41.539454 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.539408 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerStarted","Data":"0cd22d30d1a6e458a6ad3d0f3d3188e568cb2b637d914d110da833e03987822d"} Jul 07 12:43:41.540903 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.540875 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" event={"ID":"b6591db7-8228-485c-aa88-13ebba70998c","Type":"ContainerStarted","Data":"20d082629019791aab0428647451d53f41033f6b384d11789e9bf956e8658203"} Jul 07 12:43:41.543721 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.543666 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b4bc4" event={"ID":"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709","Type":"ContainerStarted","Data":"96c6f5801d389f19543266193ec8bd4b357f17fa8c7e288f56ea8a0bd87ba606"} Jul 07 12:43:41.543838 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.543745 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b4bc4" event={"ID":"9bbc4f72-0795-4ff1-b0ad-1dc68d5fb709","Type":"ContainerStarted","Data":"179f2a24aede6797742ae8fa2d833b8a5e8890cbfcc52abf2474a7d592bdaef6"} Jul 07 12:43:41.546321 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.546295 2574 generic.go:356] "Generic (PLEG): container finished" podID="6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e" containerID="20fdd37dc8bac5a129da80988f44cf33f237fa243af21818c7a19260018a2e2d" exitCode=0 Jul 07 12:43:41.546435 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.546373 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z6fdd" event={"ID":"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e","Type":"ContainerDied","Data":"20fdd37dc8bac5a129da80988f44cf33f237fa243af21818c7a19260018a2e2d"} Jul 07 12:43:41.549526 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.549495 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" event={"ID":"3644ab2c-4c37-4b02-b77c-bee617c5ed46","Type":"ContainerStarted","Data":"fc80af8490c1bd0706bac97f19bfeab359e9b0431fb832855fdc267254bb0120"} Jul 07 12:43:41.555661 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.555626 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:43:41.565122 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.565080 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b4bc4" podStartSLOduration=66.455734471 podStartE2EDuration="1m8.565066108s" totalImagesPullingTime="2.109331637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:41.558227431 +0000 UTC m=+69.002270636" watchObservedRunningTime="2026-07-07 12:43:41.565066108 +0000 UTC m=+69.009109286" Jul 07 12:43:41.599079 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.598994 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wnn8b" podStartSLOduration=1.785440805 podStartE2EDuration="3.598981871s" totalImagesPullingTime="1.813541066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:41.598875294 +0000 UTC m=+69.042918472" watchObservedRunningTime="2026-07-07 12:43:41.598981871 +0000 UTC m=+69.043025050" Jul 07 12:43:41.609387 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.609356 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75869989b6-q6q58"] Jul 07 12:43:41.630387 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.630351 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:41.699434 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.699376 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:41.761784 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:41.761595 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:42.547320 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.547283 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-559b4857cc-ktlxm"] Jul 07 12:43:42.550720 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.550047 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.553842 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.553807 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ql9zz\"" Jul 07 12:43:42.553971 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.553896 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 12:43:42.554656 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.554162 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 12:43:42.554656 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.554234 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 12:43:42.554656 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.554460 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4292dul3v1290\"" Jul 07 12:43:42.554656 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.554489 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 12:43:42.559235 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.559182 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-559b4857cc-ktlxm"] Jul 07 12:43:42.669354 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.669300 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f06fe04d-b6bc-4540-ad4a-974aaa983a22-secret-metrics-server-client-certs\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.669354 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.669347 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f06fe04d-b6bc-4540-ad4a-974aaa983a22-secret-metrics-server-tls\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.669607 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.669384 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f06fe04d-b6bc-4540-ad4a-974aaa983a22-metrics-server-audit-profiles\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.669607 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.669469 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz7r2\" (UniqueName: \"kubernetes.io/projected/f06fe04d-b6bc-4540-ad4a-974aaa983a22-kube-api-access-sz7r2\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.669607 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.669511 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f06fe04d-b6bc-4540-ad4a-974aaa983a22-client-ca-bundle\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.669780 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.669739 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f06fe04d-b6bc-4540-ad4a-974aaa983a22-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.670057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.670034 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f06fe04d-b6bc-4540-ad4a-974aaa983a22-audit-log\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.771593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.771553 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f06fe04d-b6bc-4540-ad4a-974aaa983a22-secret-metrics-server-client-certs\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.771809 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.771598 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f06fe04d-b6bc-4540-ad4a-974aaa983a22-secret-metrics-server-tls\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.771809 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.771761 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f06fe04d-b6bc-4540-ad4a-974aaa983a22-metrics-server-audit-profiles\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.771929 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.771818 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz7r2\" (UniqueName: \"kubernetes.io/projected/f06fe04d-b6bc-4540-ad4a-974aaa983a22-kube-api-access-sz7r2\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.771929 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.771849 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f06fe04d-b6bc-4540-ad4a-974aaa983a22-client-ca-bundle\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.771929 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.771891 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f06fe04d-b6bc-4540-ad4a-974aaa983a22-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.772079 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.771939 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f06fe04d-b6bc-4540-ad4a-974aaa983a22-audit-log\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.773573 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.772329 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f06fe04d-b6bc-4540-ad4a-974aaa983a22-audit-log\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.773724 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.773660 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f06fe04d-b6bc-4540-ad4a-974aaa983a22-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.774590 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.774545 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f06fe04d-b6bc-4540-ad4a-974aaa983a22-metrics-server-audit-profiles\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.775651 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.775607 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f06fe04d-b6bc-4540-ad4a-974aaa983a22-secret-metrics-server-client-certs\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.776346 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.776307 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f06fe04d-b6bc-4540-ad4a-974aaa983a22-secret-metrics-server-tls\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.777056 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.777029 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f06fe04d-b6bc-4540-ad4a-974aaa983a22-client-ca-bundle\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.781089 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.781069 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz7r2\" (UniqueName: \"kubernetes.io/projected/f06fe04d-b6bc-4540-ad4a-974aaa983a22-kube-api-access-sz7r2\") pod \"metrics-server-559b4857cc-ktlxm\" (UID: \"f06fe04d-b6bc-4540-ad4a-974aaa983a22\") " pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:42.864067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:42.863966 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:43:43.008235 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.008198 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:43.014578 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.014545 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t"] Jul 07 12:43:43.017186 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.017140 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" Jul 07 12:43:43.020016 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.019777 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 12:43:43.020016 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.019779 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-sw2bk\"" Jul 07 12:43:43.031142 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.030868 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t"] Jul 07 12:43:43.088869 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.088836 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:43.176804 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.176721 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6c4f650c-ff41-433e-abdc-fc6503aee125-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-96x9t\" (UID: \"6c4f650c-ff41-433e-abdc-fc6503aee125\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" Jul 07 12:43:43.176963 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.176844 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq9gm\" (UniqueName: \"kubernetes.io/projected/6c4f650c-ff41-433e-abdc-fc6503aee125-kube-api-access-jq9gm\") pod \"monitoring-plugin-d5dcff7cf-96x9t\" (UID: \"6c4f650c-ff41-433e-abdc-fc6503aee125\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" Jul 07 12:43:43.278342 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.278307 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq9gm\" (UniqueName: \"kubernetes.io/projected/6c4f650c-ff41-433e-abdc-fc6503aee125-kube-api-access-jq9gm\") pod \"monitoring-plugin-d5dcff7cf-96x9t\" (UID: \"6c4f650c-ff41-433e-abdc-fc6503aee125\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" Jul 07 12:43:43.278491 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.278370 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6c4f650c-ff41-433e-abdc-fc6503aee125-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-96x9t\" (UID: \"6c4f650c-ff41-433e-abdc-fc6503aee125\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" Jul 07 12:43:43.278539 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:43.278489 2574 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 07 12:43:43.278583 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:43:43.278557 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c4f650c-ff41-433e-abdc-fc6503aee125-monitoring-plugin-cert podName:6c4f650c-ff41-433e-abdc-fc6503aee125 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:43.778535151 +0000 UTC m=+71.222578310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/6c4f650c-ff41-433e-abdc-fc6503aee125-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-96x9t" (UID: "6c4f650c-ff41-433e-abdc-fc6503aee125") : secret "monitoring-plugin-cert" not found Jul 07 12:43:43.287640 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.287607 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq9gm\" (UniqueName: \"kubernetes.io/projected/6c4f650c-ff41-433e-abdc-fc6503aee125-kube-api-access-jq9gm\") pod \"monitoring-plugin-d5dcff7cf-96x9t\" (UID: \"6c4f650c-ff41-433e-abdc-fc6503aee125\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" Jul 07 12:43:43.722834 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.722796 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:43.781663 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.781627 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:43.783728 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.783678 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6c4f650c-ff41-433e-abdc-fc6503aee125-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-96x9t\" (UID: \"6c4f650c-ff41-433e-abdc-fc6503aee125\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" Jul 07 12:43:43.786353 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.786330 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6c4f650c-ff41-433e-abdc-fc6503aee125-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-96x9t\" (UID: \"6c4f650c-ff41-433e-abdc-fc6503aee125\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" Jul 07 12:43:43.931868 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:43.931848 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" Jul 07 12:43:44.331092 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.330227 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-559b4857cc-ktlxm"] Jul 07 12:43:44.487073 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.487036 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:43:44.492102 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.492066 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.494819 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.494794 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 12:43:44.495008 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.494879 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 12:43:44.495408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.495135 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 12:43:44.495408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.495226 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 12:43:44.495408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.495234 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 12:43:44.495408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.495228 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 12:43:44.495408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.495336 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d5ij900o8bqe6\"" Jul 07 12:43:44.495733 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.495594 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 12:43:44.496391 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.496237 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 12:43:44.498270 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.496500 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 12:43:44.498270 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.498015 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 12:43:44.498270 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.498117 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 12:43:44.498468 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.498312 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nvbjk\"" Jul 07 12:43:44.498881 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.498546 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 12:43:44.498881 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.498600 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:44.498881 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.498625 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 12:43:44.500317 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.500169 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 12:43:44.512721 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.512678 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:43:44.591860 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.591824 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.591860 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.591866 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-config\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592099 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.591893 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592099 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.591975 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592099 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592016 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592099 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592077 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592119 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw67m\" (UniqueName: \"kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-kube-api-access-qw67m\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592151 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592170 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592206 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592240 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592285 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592306 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592624 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592324 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592624 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592339 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592624 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592362 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592624 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592383 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592624 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592405 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592624 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.592624 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.592441 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.693966 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.693928 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.693966 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.693971 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-config\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.693998 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694033 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694053 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694089 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694115 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw67m\" (UniqueName: \"kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-kube-api-access-qw67m\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694142 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694160 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694194 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694212 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694231 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694245 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694268 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694285 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694304 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694320 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694349 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694366 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.694569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694383 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.695059 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.694843 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.695155 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.695125 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.696261 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.695887 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.696261 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.695907 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.696261 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.695943 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.697559 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.697117 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.697742 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.697586 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.697742 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.697668 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.700157 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.700134 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.700523 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.700320 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.701718 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.701295 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-config\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.701718 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.701461 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.701718 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.701512 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.701718 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.701665 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.702759 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.702732 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.703543 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.703518 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.703924 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.703905 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.704438 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.704400 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.704884 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.704862 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.705154 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.705133 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw67m\" (UniqueName: \"kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-kube-api-access-qw67m\") pod \"prometheus-k8s-0\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.805156 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:44.805077 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:44.849990 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:44.849957 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf06fe04d_b6bc_4540_ad4a_974aaa983a22.slice/crio-d852210ddd1ee8ab306447ba2467257df6d5251d3bafef5294e01cbb36b6b47b WatchSource:0}: Error finding container d852210ddd1ee8ab306447ba2467257df6d5251d3bafef5294e01cbb36b6b47b: Status 404 returned error can't find the container with id d852210ddd1ee8ab306447ba2467257df6d5251d3bafef5294e01cbb36b6b47b Jul 07 12:43:45.068341 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.068262 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t"] Jul 07 12:43:45.091944 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:43:45.091903 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4f650c_ff41_433e_abdc_fc6503aee125.slice/crio-d88b98dc81a383682ba92eed5134307be7c6de2ad0911b5491252d10035671c4 WatchSource:0}: Error finding container d88b98dc81a383682ba92eed5134307be7c6de2ad0911b5491252d10035671c4: Status 404 returned error can't find the container with id d88b98dc81a383682ba92eed5134307be7c6de2ad0911b5491252d10035671c4 Jul 07 12:43:45.154428 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.154326 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:43:45.194883 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.194853 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:45.287533 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.287101 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:45.386612 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.386580 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:45.509934 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.509870 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vfdz6" Jul 07 12:43:45.572870 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.572786 2574 generic.go:356] "Generic (PLEG): container finished" podID="23deede0-66e8-4510-87b3-e529f030053f" containerID="4c8d6fb9c33b045c55105d6b7832ba2b7a7d7a7899e4d8ec3756f59dd50eaa5b" exitCode=0 Jul 07 12:43:45.573000 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.572877 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerDied","Data":"4c8d6fb9c33b045c55105d6b7832ba2b7a7d7a7899e4d8ec3756f59dd50eaa5b"} Jul 07 12:43:45.575935 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.575908 2574 generic.go:356] "Generic (PLEG): container finished" podID="5177becb-9382-4534-b46e-50526b9a519c" containerID="319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597" exitCode=0 Jul 07 12:43:45.575995 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.575982 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerDied","Data":"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597"} Jul 07 12:43:45.576215 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.576043 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerStarted","Data":"4d2df64f376b1ea7529aa3572396a7c593dda24d70ce795d38aebf8ddbedb37c"} Jul 07 12:43:45.577911 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.577835 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" event={"ID":"6c4f650c-ff41-433e-abdc-fc6503aee125","Type":"ContainerStarted","Data":"d88b98dc81a383682ba92eed5134307be7c6de2ad0911b5491252d10035671c4"} Jul 07 12:43:45.584660 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.583952 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" event={"ID":"ce62cb92-033c-4abc-a117-b3896d689f23","Type":"ContainerStarted","Data":"fede5514ef376332bfe119104ce95ed212b0863791655a799978ed8aca0c8dbe"} Jul 07 12:43:45.584660 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.584050 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" event={"ID":"ce62cb92-033c-4abc-a117-b3896d689f23","Type":"ContainerStarted","Data":"1503cb04fbb9b1445f57aca0c6ac7bb4bc5d4d3ee32453ded3b9f11be7dd130e"} Jul 07 12:43:45.584660 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.584082 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" event={"ID":"ce62cb92-033c-4abc-a117-b3896d689f23","Type":"ContainerStarted","Data":"a7042adb7ee0e8e68585d5a8fe848778a9c2b486ede2789f30e304dfec6f76ab"} Jul 07 12:43:45.588073 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.587912 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" event={"ID":"b6591db7-8228-485c-aa88-13ebba70998c","Type":"ContainerStarted","Data":"06b0364794139d53fd8bf67d99ed72a51e3e34423e8f5373b4f562125d2984e5"} Jul 07 12:43:45.588073 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.587980 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" event={"ID":"b6591db7-8228-485c-aa88-13ebba70998c","Type":"ContainerStarted","Data":"0aa84169b08bb0a8fe70f98771d26e8c709a33f25106f62bd09abd016a5ec62a"} Jul 07 12:43:45.588073 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.588019 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" event={"ID":"b6591db7-8228-485c-aa88-13ebba70998c","Type":"ContainerStarted","Data":"9dea50a1a0df6986893ef2e39fe962353b3596e6a95da21876c6c14935214538"} Jul 07 12:43:45.589546 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.589479 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" event={"ID":"f06fe04d-b6bc-4540-ad4a-974aaa983a22","Type":"ContainerStarted","Data":"d852210ddd1ee8ab306447ba2467257df6d5251d3bafef5294e01cbb36b6b47b"} Jul 07 12:43:45.592324 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.591769 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w4grp" event={"ID":"ba088306-b1bd-4a02-9ee2-e547cc187b9f","Type":"ContainerStarted","Data":"beb15f4a47d5803b92d4fd3bd88b86a040136d122352b6947b356a7458e4b7da"} Jul 07 12:43:45.592324 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.592248 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:43:45.596808 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.596760 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cbl98" event={"ID":"5e3e1d99-1fd1-4f01-980b-146f14cb0b43","Type":"ContainerStarted","Data":"d77ba3ed8724937ebed59ee157cfb9bfd3ce7f0dbbf234a9812dbd3c107e4cba"} Jul 07 12:43:45.605153 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.605124 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z6fdd" event={"ID":"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e","Type":"ContainerStarted","Data":"ad5a9aa8ae1caf1cfd7d0c19635bac3533645da31e13d823cbab8b54beb426f0"} Jul 07 12:43:45.605259 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.605170 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z6fdd" event={"ID":"6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e","Type":"ContainerStarted","Data":"a3bc8ef65f04ac6ffda25b57f42e4d633c0eaae3b5bea7d7dd0725454fd1aa39"} Jul 07 12:43:45.615643 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.615586 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cbl98" podStartSLOduration=66.224316722 podStartE2EDuration="1m12.615571743s" totalImagesPullingTime="6.391255021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:45.614246947 +0000 UTC m=+73.058290128" watchObservedRunningTime="2026-07-07 12:43:45.615571743 +0000 UTC m=+73.059614900" Jul 07 12:43:45.630108 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.630050 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-w4grp" podStartSLOduration=66.026741233 podStartE2EDuration="1m12.630033859s" totalImagesPullingTime="6.603292626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:45.629113328 +0000 UTC m=+73.073156508" watchObservedRunningTime="2026-07-07 12:43:45.630033859 +0000 UTC m=+73.074077037" Jul 07 12:43:45.682858 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.682788 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gr8cr" podStartSLOduration=2.242783042 podStartE2EDuration="7.682766459s" totalImagesPullingTime="5.439983417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:45.647631176 +0000 UTC m=+73.091674358" watchObservedRunningTime="2026-07-07 12:43:45.682766459 +0000 UTC m=+73.126809640" Jul 07 12:43:45.705127 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.705076 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-z6fdd" podStartSLOduration=6.198193514 podStartE2EDuration="7.705061427s" totalImagesPullingTime="1.390946232s" totalInitContainerRuntime="115.921681ms" isStatefulPod=true podCreationTimestamp="2026-07-07 12:43:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:45.704289755 +0000 UTC m=+73.148332960" watchObservedRunningTime="2026-07-07 12:43:45.705061427 +0000 UTC m=+73.149104606" Jul 07 12:43:45.720792 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.720761 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:45.805873 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:45.805830 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:49.155135 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.155110 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:49.256884 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.256852 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:49.639967 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.639593 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerStarted","Data":"4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea"} Jul 07 12:43:49.639967 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.639731 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerStarted","Data":"613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2"} Jul 07 12:43:49.639967 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.639784 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerStarted","Data":"dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f"} Jul 07 12:43:49.639967 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.639824 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerStarted","Data":"28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c"} Jul 07 12:43:49.639967 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.639863 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerStarted","Data":"d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a"} Jul 07 12:43:49.651018 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.650961 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerStarted","Data":"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173"} Jul 07 12:43:49.651150 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.651049 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerStarted","Data":"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b"} Jul 07 12:43:49.651150 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.651090 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerStarted","Data":"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2"} Jul 07 12:43:49.651150 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.651131 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerStarted","Data":"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737"} Jul 07 12:43:49.651308 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.651166 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerStarted","Data":"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba"} Jul 07 12:43:49.654437 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.654370 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" event={"ID":"6c4f650c-ff41-433e-abdc-fc6503aee125","Type":"ContainerStarted","Data":"6b0f633512c9a97a8c4af3dd2cd22c9c0973d86cbdeafb7edb2579d91f951cba"} Jul 07 12:43:49.654911 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.654861 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" Jul 07 12:43:49.660139 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.659868 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" event={"ID":"b6591db7-8228-485c-aa88-13ebba70998c","Type":"ContainerStarted","Data":"d86df0120ae1da14b79eaf5c66065823cb75a9c1bbf5a7f02f6d9242de4a1936"} Jul 07 12:43:49.660139 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.659928 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" event={"ID":"b6591db7-8228-485c-aa88-13ebba70998c","Type":"ContainerStarted","Data":"154a30b4200da6c26acf86fc4d87cb8d305fc2dd7ff7977396b38a457f2a2d84"} Jul 07 12:43:49.660139 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.659969 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" event={"ID":"b6591db7-8228-485c-aa88-13ebba70998c","Type":"ContainerStarted","Data":"33226b211ab8548344126d14700e31468da45e575b405735ded79dd2517a3169"} Jul 07 12:43:49.660139 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.660015 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:49.662940 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.662916 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" Jul 07 12:43:49.672393 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.667549 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" event={"ID":"f06fe04d-b6bc-4540-ad4a-974aaa983a22","Type":"ContainerStarted","Data":"3ad1221bd878b4f5ab14cf32b5ba2a8fe413a66d9e89b4cb2caa7f95756e1dfe"} Jul 07 12:43:49.672763 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.672431 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-96x9t" podStartSLOduration=3.646573788 podStartE2EDuration="7.672413938s" totalImagesPullingTime="4.02584015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:49.670511298 +0000 UTC m=+77.114554478" watchObservedRunningTime="2026-07-07 12:43:49.672413938 +0000 UTC m=+77.116457118" Jul 07 12:43:49.693741 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.693673 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" podStartSLOduration=3.436619563 podStartE2EDuration="7.693657039s" totalImagesPullingTime="4.257037476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:49.692151949 +0000 UTC m=+77.136195356" watchObservedRunningTime="2026-07-07 12:43:49.693657039 +0000 UTC m=+77.137700221" Jul 07 12:43:49.738498 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:49.738437 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" podStartSLOduration=2.2710908659999998 podStartE2EDuration="9.738416517s" totalImagesPullingTime="7.467325651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:49.737135082 +0000 UTC m=+77.181178265" watchObservedRunningTime="2026-07-07 12:43:49.738416517 +0000 UTC m=+77.182459696" Jul 07 12:43:50.675273 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:50.675239 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerStarted","Data":"8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346"} Jul 07 12:43:50.678705 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:50.678666 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerStarted","Data":"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154"} Jul 07 12:43:50.684528 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:50.684508 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6686974f76-rmcwm" Jul 07 12:43:50.707003 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:50.706956 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.030472207 podStartE2EDuration="11.706941353s" totalImagesPullingTime="7.648187548s" totalInitContainerRuntime="28.281598ms" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:50.705488906 +0000 UTC m=+78.149532114" watchObservedRunningTime="2026-07-07 12:43:50.706941353 +0000 UTC m=+78.150984531" Jul 07 12:43:50.737743 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:50.737677 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.105583148 podStartE2EDuration="6.737660064s" totalImagesPullingTime="3.604157398s" totalInitContainerRuntime="27.919518ms" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:50.735492517 +0000 UTC m=+78.179535698" watchObservedRunningTime="2026-07-07 12:43:50.737660064 +0000 UTC m=+78.181703242" Jul 07 12:43:51.096785 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:51.096740 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:51.205391 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:51.205361 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:51.571625 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:51.571599 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:52.745796 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:52.745752 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:52.844266 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:52.844222 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:53.807063 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:53.807028 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:53.865901 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:53.865872 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:54.701284 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:54.701169 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xvjb2" event={"ID":"f5da8ea0-29bd-4abf-803d-1f76999d78a3","Type":"ContainerStarted","Data":"0c3334babaa2daa5005855be47bfc0c1788b6c5634037f02239d791105c6ee34"} Jul 07 12:43:54.722120 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:54.722056 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xvjb2" podStartSLOduration=1.9253950990000002 podStartE2EDuration="27.722038044s" totalImagesPullingTime="25.796642945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:43:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-07 12:43:54.719422212 +0000 UTC m=+82.163465392" watchObservedRunningTime="2026-07-07 12:43:54.722038044 +0000 UTC m=+82.166081224" Jul 07 12:43:54.789726 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:54.789669 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:54.805659 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:54.805633 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:43:54.846737 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:54.846706 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:56.204929 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:56.204893 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c7cc86965-hll8s"] Jul 07 12:43:56.324168 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:56.324134 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:56.421319 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:56.421288 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:43:56.521622 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:43:56.521545 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:44:01.742971 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:01.742935 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:44:01.802160 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:01.802125 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:44:02.864876 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:02.864839 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:44:02.865241 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:02.864904 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:44:06.639450 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:06.639408 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-75869989b6-q6q58" podUID="ceaa562d-5f18-410c-b672-36a6384fcbb5" containerName="console" containerID="cri-o://ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf" gracePeriod=15 Jul 07 12:44:07.404436 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.404406 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:44:07.522578 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.522480 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-service-ca\") pod \"ceaa562d-5f18-410c-b672-36a6384fcbb5\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " Jul 07 12:44:07.522578 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.522561 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-oauth-serving-cert\") pod \"ceaa562d-5f18-410c-b672-36a6384fcbb5\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " Jul 07 12:44:07.522829 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.522611 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-serving-cert\") pod \"ceaa562d-5f18-410c-b672-36a6384fcbb5\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " Jul 07 12:44:07.522829 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.522637 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-oauth-config\") pod \"ceaa562d-5f18-410c-b672-36a6384fcbb5\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " Jul 07 12:44:07.522829 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.522713 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47p6n\" (UniqueName: \"kubernetes.io/projected/ceaa562d-5f18-410c-b672-36a6384fcbb5-kube-api-access-47p6n\") pod \"ceaa562d-5f18-410c-b672-36a6384fcbb5\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " Jul 07 12:44:07.522829 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.522747 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-config\") pod \"ceaa562d-5f18-410c-b672-36a6384fcbb5\" (UID: \"ceaa562d-5f18-410c-b672-36a6384fcbb5\") " Jul 07 12:44:07.523013 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.522930 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-service-ca" (OuterVolumeSpecName: "service-ca") pod "ceaa562d-5f18-410c-b672-36a6384fcbb5" (UID: "ceaa562d-5f18-410c-b672-36a6384fcbb5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:44:07.523013 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.522985 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ceaa562d-5f18-410c-b672-36a6384fcbb5" (UID: "ceaa562d-5f18-410c-b672-36a6384fcbb5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:44:07.523118 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.523046 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-service-ca\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:07.523257 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.523236 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-config" (OuterVolumeSpecName: "console-config") pod "ceaa562d-5f18-410c-b672-36a6384fcbb5" (UID: "ceaa562d-5f18-410c-b672-36a6384fcbb5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:44:07.525121 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.525086 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ceaa562d-5f18-410c-b672-36a6384fcbb5" (UID: "ceaa562d-5f18-410c-b672-36a6384fcbb5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:07.525224 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.525090 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceaa562d-5f18-410c-b672-36a6384fcbb5-kube-api-access-47p6n" (OuterVolumeSpecName: "kube-api-access-47p6n") pod "ceaa562d-5f18-410c-b672-36a6384fcbb5" (UID: "ceaa562d-5f18-410c-b672-36a6384fcbb5"). InnerVolumeSpecName "kube-api-access-47p6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:44:07.525224 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.525152 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ceaa562d-5f18-410c-b672-36a6384fcbb5" (UID: "ceaa562d-5f18-410c-b672-36a6384fcbb5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:07.623707 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.623664 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-serving-cert\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:07.623863 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.623718 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-oauth-config\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:07.623863 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.623728 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47p6n\" (UniqueName: \"kubernetes.io/projected/ceaa562d-5f18-410c-b672-36a6384fcbb5-kube-api-access-47p6n\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:07.623863 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.623737 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-console-config\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:07.623863 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.623746 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ceaa562d-5f18-410c-b672-36a6384fcbb5-oauth-serving-cert\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:07.761018 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.760980 2574 generic.go:356] "Generic (PLEG): container finished" podID="ceaa562d-5f18-410c-b672-36a6384fcbb5" containerID="ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf" exitCode=0 Jul 07 12:44:07.761471 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.761041 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75869989b6-q6q58" Jul 07 12:44:07.761471 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.761045 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75869989b6-q6q58" event={"ID":"ceaa562d-5f18-410c-b672-36a6384fcbb5","Type":"ContainerDied","Data":"ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf"} Jul 07 12:44:07.761471 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.761174 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75869989b6-q6q58" event={"ID":"ceaa562d-5f18-410c-b672-36a6384fcbb5","Type":"ContainerDied","Data":"cccc65f5e8f512567ab4e1e40095ad4f75aa98b90697dc7ed9dc62344406663d"} Jul 07 12:44:07.761471 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.761195 2574 scope.go:117] "RemoveContainer" containerID="ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf" Jul 07 12:44:07.770553 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.770535 2574 scope.go:117] "RemoveContainer" containerID="ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf" Jul 07 12:44:07.770846 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:44:07.770828 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf\": container with ID starting with ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf not found: ID does not exist" containerID="ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf" Jul 07 12:44:07.770914 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.770857 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf"} err="failed to get container status \"ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf\": rpc error: code = NotFound desc = could not find container \"ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf\": container with ID starting with ac9b44d96e924689e0bcdbc3d1d716910e1af7fe191e91cbac2607dcd548faaf not found: ID does not exist" Jul 07 12:44:07.782716 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.782647 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75869989b6-q6q58"] Jul 07 12:44:07.785668 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:07.785647 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75869989b6-q6q58"] Jul 07 12:44:08.793986 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:08.793953 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:44:08.847768 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:08.847724 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:44:09.188279 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:09.188207 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceaa562d-5f18-410c-b672-36a6384fcbb5" path="/var/lib/kubelet/pods/ceaa562d-5f18-410c-b672-36a6384fcbb5/volumes" Jul 07 12:44:09.935950 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:09.935916 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:44:10.036901 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:10.036870 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:44:17.619819 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:17.619781 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-w4grp" Jul 07 12:44:21.225067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.225026 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7c7cc86965-hll8s" podUID="ed4e2b39-79c5-4fbd-addc-2277488c6160" containerName="console" containerID="cri-o://470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683" gracePeriod=15 Jul 07 12:44:21.495483 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.495421 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:44:21.543599 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.543562 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-trusted-ca-bundle\") pod \"ed4e2b39-79c5-4fbd-addc-2277488c6160\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " Jul 07 12:44:21.543781 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.543617 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-oauth-config\") pod \"ed4e2b39-79c5-4fbd-addc-2277488c6160\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " Jul 07 12:44:21.543781 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.543639 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-serving-cert\") pod \"ed4e2b39-79c5-4fbd-addc-2277488c6160\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " Jul 07 12:44:21.543781 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.543660 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2qxw\" (UniqueName: \"kubernetes.io/projected/ed4e2b39-79c5-4fbd-addc-2277488c6160-kube-api-access-z2qxw\") pod \"ed4e2b39-79c5-4fbd-addc-2277488c6160\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " Jul 07 12:44:21.543781 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.543684 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-service-ca\") pod \"ed4e2b39-79c5-4fbd-addc-2277488c6160\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " Jul 07 12:44:21.543781 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.543729 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-config\") pod \"ed4e2b39-79c5-4fbd-addc-2277488c6160\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " Jul 07 12:44:21.544025 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.543881 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-oauth-serving-cert\") pod \"ed4e2b39-79c5-4fbd-addc-2277488c6160\" (UID: \"ed4e2b39-79c5-4fbd-addc-2277488c6160\") " Jul 07 12:44:21.544137 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.544106 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ed4e2b39-79c5-4fbd-addc-2277488c6160" (UID: "ed4e2b39-79c5-4fbd-addc-2277488c6160"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:44:21.544238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.544157 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-service-ca" (OuterVolumeSpecName: "service-ca") pod "ed4e2b39-79c5-4fbd-addc-2277488c6160" (UID: "ed4e2b39-79c5-4fbd-addc-2277488c6160"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:44:21.544238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.544213 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-trusted-ca-bundle\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:21.544379 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.544352 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ed4e2b39-79c5-4fbd-addc-2277488c6160" (UID: "ed4e2b39-79c5-4fbd-addc-2277488c6160"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:44:21.544559 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.544537 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-config" (OuterVolumeSpecName: "console-config") pod "ed4e2b39-79c5-4fbd-addc-2277488c6160" (UID: "ed4e2b39-79c5-4fbd-addc-2277488c6160"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:44:21.545966 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.545939 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ed4e2b39-79c5-4fbd-addc-2277488c6160" (UID: "ed4e2b39-79c5-4fbd-addc-2277488c6160"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:21.546069 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.545981 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed4e2b39-79c5-4fbd-addc-2277488c6160-kube-api-access-z2qxw" (OuterVolumeSpecName: "kube-api-access-z2qxw") pod "ed4e2b39-79c5-4fbd-addc-2277488c6160" (UID: "ed4e2b39-79c5-4fbd-addc-2277488c6160"). InnerVolumeSpecName "kube-api-access-z2qxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:44:21.546069 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.546011 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ed4e2b39-79c5-4fbd-addc-2277488c6160" (UID: "ed4e2b39-79c5-4fbd-addc-2277488c6160"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:21.644707 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.644671 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-config\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:21.644707 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.644714 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-oauth-serving-cert\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:21.644883 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.644724 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-oauth-config\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:21.644883 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.644733 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed4e2b39-79c5-4fbd-addc-2277488c6160-console-serving-cert\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:21.644883 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.644742 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2qxw\" (UniqueName: \"kubernetes.io/projected/ed4e2b39-79c5-4fbd-addc-2277488c6160-kube-api-access-z2qxw\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:21.644883 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.644750 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed4e2b39-79c5-4fbd-addc-2277488c6160-service-ca\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:21.814802 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.814720 2574 generic.go:356] "Generic (PLEG): container finished" podID="ed4e2b39-79c5-4fbd-addc-2277488c6160" containerID="470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683" exitCode=0 Jul 07 12:44:21.814802 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.814786 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7cc86965-hll8s" event={"ID":"ed4e2b39-79c5-4fbd-addc-2277488c6160","Type":"ContainerDied","Data":"470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683"} Jul 07 12:44:21.814992 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.814815 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7cc86965-hll8s" Jul 07 12:44:21.814992 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.814822 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7cc86965-hll8s" event={"ID":"ed4e2b39-79c5-4fbd-addc-2277488c6160","Type":"ContainerDied","Data":"6b72f45754e60df83bf4a0afb2ed2de421cc90b51cc6f857ad41168c3a45be40"} Jul 07 12:44:21.814992 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.814842 2574 scope.go:117] "RemoveContainer" containerID="470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683" Jul 07 12:44:21.823978 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.823960 2574 scope.go:117] "RemoveContainer" containerID="470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683" Jul 07 12:44:21.824302 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:44:21.824283 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683\": container with ID starting with 470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683 not found: ID does not exist" containerID="470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683" Jul 07 12:44:21.824358 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.824310 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683"} err="failed to get container status \"470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683\": rpc error: code = NotFound desc = could not find container \"470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683\": container with ID starting with 470ee984c4afcb86203a223b240b126f70cb5a0e120c2a243b2bd6d71880c683 not found: ID does not exist" Jul 07 12:44:21.835489 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.835463 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c7cc86965-hll8s"] Jul 07 12:44:21.838735 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:21.838716 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c7cc86965-hll8s"] Jul 07 12:44:22.869316 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:22.869280 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:44:22.873037 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:22.873018 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-559b4857cc-ktlxm" Jul 07 12:44:23.187980 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:23.187904 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed4e2b39-79c5-4fbd-addc-2277488c6160" path="/var/lib/kubelet/pods/ed4e2b39-79c5-4fbd-addc-2277488c6160/volumes" Jul 07 12:44:44.805286 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:44.805247 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:44:44.824462 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:44.824435 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:44:44.925608 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:44.925575 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:44:57.517904 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:57.517853 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:44:57.799330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:57.799217 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:44:58.582303 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.582264 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:44:58.582782 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.582529 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="alertmanager" containerID="cri-o://d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a" gracePeriod=120 Jul 07 12:44:58.582782 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.582601 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy-metric" containerID="cri-o://4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea" gracePeriod=120 Jul 07 12:44:58.582782 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.582628 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy-web" containerID="cri-o://dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f" gracePeriod=120 Jul 07 12:44:58.582782 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.582672 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="config-reloader" containerID="cri-o://28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c" gracePeriod=120 Jul 07 12:44:58.582782 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.582714 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy" containerID="cri-o://613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2" gracePeriod=120 Jul 07 12:44:58.582782 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.582643 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="prom-label-proxy" containerID="cri-o://8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346" gracePeriod=120 Jul 07 12:44:58.967736 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.967706 2574 generic.go:356] "Generic (PLEG): container finished" podID="23deede0-66e8-4510-87b3-e529f030053f" containerID="8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346" exitCode=0 Jul 07 12:44:58.967736 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.967734 2574 generic.go:356] "Generic (PLEG): container finished" podID="23deede0-66e8-4510-87b3-e529f030053f" containerID="4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea" exitCode=0 Jul 07 12:44:58.967736 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.967740 2574 generic.go:356] "Generic (PLEG): container finished" podID="23deede0-66e8-4510-87b3-e529f030053f" containerID="613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2" exitCode=0 Jul 07 12:44:58.967736 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.967745 2574 generic.go:356] "Generic (PLEG): container finished" podID="23deede0-66e8-4510-87b3-e529f030053f" containerID="28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c" exitCode=0 Jul 07 12:44:58.967978 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.967751 2574 generic.go:356] "Generic (PLEG): container finished" podID="23deede0-66e8-4510-87b3-e529f030053f" containerID="d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a" exitCode=0 Jul 07 12:44:58.967978 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.967715 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerDied","Data":"8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346"} Jul 07 12:44:58.967978 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.967819 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerDied","Data":"4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea"} Jul 07 12:44:58.967978 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.967849 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerDied","Data":"613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2"} Jul 07 12:44:58.967978 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.967900 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerDied","Data":"28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c"} Jul 07 12:44:58.967978 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:58.967926 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerDied","Data":"d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a"} Jul 07 12:44:59.825729 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.825685 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:44:59.973257 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973231 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973445 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973268 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-web-config\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973445 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973300 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-trusted-ca-bundle\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973445 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973321 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973445 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973359 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk9mm\" (UniqueName: \"kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-kube-api-access-fk9mm\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973445 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973386 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-cluster-tls-config\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973445 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973427 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-tls-assets\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973800 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973471 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-main-tls\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973800 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973517 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-web\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973800 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973547 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-metrics-client-ca\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973800 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973572 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-config-out\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973800 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973597 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-main-db\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973800 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973657 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-config-volume\") pod \"23deede0-66e8-4510-87b3-e529f030053f\" (UID: \"23deede0-66e8-4510-87b3-e529f030053f\") " Jul 07 12:44:59.973800 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973673 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:44:59.974170 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.973948 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:44:59.974609 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.974573 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:44:59.975440 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.975412 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:44:59.976226 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.976198 2574 generic.go:356] "Generic (PLEG): container finished" podID="23deede0-66e8-4510-87b3-e529f030053f" containerID="dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f" exitCode=0 Jul 07 12:44:59.976335 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.976295 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerDied","Data":"dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f"} Jul 07 12:44:59.976335 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.976308 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:44:59.976439 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.976358 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23deede0-66e8-4510-87b3-e529f030053f","Type":"ContainerDied","Data":"0cd22d30d1a6e458a6ad3d0f3d3188e568cb2b637d914d110da833e03987822d"} Jul 07 12:44:59.976439 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.976401 2574 scope.go:117] "RemoveContainer" containerID="8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346" Jul 07 12:44:59.977458 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.977411 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:59.977595 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.977568 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-kube-api-access-fk9mm" (OuterVolumeSpecName: "kube-api-access-fk9mm") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "kube-api-access-fk9mm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:44:59.977917 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.977867 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-config-volume" (OuterVolumeSpecName: "config-volume") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:59.978038 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.978017 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:59.978509 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.978478 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:59.978598 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.978556 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-config-out" (OuterVolumeSpecName: "config-out") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:44:59.978979 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.978951 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:59.979081 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.979058 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:44:59.982508 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.982487 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:59.988758 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.988737 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-web-config" (OuterVolumeSpecName: "web-config") pod "23deede0-66e8-4510-87b3-e529f030053f" (UID: "23deede0-66e8-4510-87b3-e529f030053f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:59.995841 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:44:59.995817 2574 scope.go:117] "RemoveContainer" containerID="4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea" Jul 07 12:45:00.002422 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.002405 2574 scope.go:117] "RemoveContainer" containerID="613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2" Jul 07 12:45:00.008788 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.008769 2574 scope.go:117] "RemoveContainer" containerID="dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f" Jul 07 12:45:00.014924 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.014907 2574 scope.go:117] "RemoveContainer" containerID="28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c" Jul 07 12:45:00.021287 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.021269 2574 scope.go:117] "RemoveContainer" containerID="d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a" Jul 07 12:45:00.028067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.028051 2574 scope.go:117] "RemoveContainer" containerID="4c8d6fb9c33b045c55105d6b7832ba2b7a7d7a7899e4d8ec3756f59dd50eaa5b" Jul 07 12:45:00.034537 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.034517 2574 scope.go:117] "RemoveContainer" containerID="8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346" Jul 07 12:45:00.034799 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:00.034778 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346\": container with ID starting with 8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346 not found: ID does not exist" containerID="8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346" Jul 07 12:45:00.034860 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.034807 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346"} err="failed to get container status \"8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346\": rpc error: code = NotFound desc = could not find container \"8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346\": container with ID starting with 8bea89f8c1093e2028ac66035f9660fbd35f200e41d068e2ef4e9502aebe3346 not found: ID does not exist" Jul 07 12:45:00.034860 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.034824 2574 scope.go:117] "RemoveContainer" containerID="4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea" Jul 07 12:45:00.035027 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:00.035013 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea\": container with ID starting with 4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea not found: ID does not exist" containerID="4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea" Jul 07 12:45:00.035074 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.035029 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea"} err="failed to get container status \"4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea\": rpc error: code = NotFound desc = could not find container \"4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea\": container with ID starting with 4c68103a1de2aebb7a24279c465a095b27bba457255bedd076a17d58afa90eea not found: ID does not exist" Jul 07 12:45:00.035074 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.035042 2574 scope.go:117] "RemoveContainer" containerID="613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2" Jul 07 12:45:00.035240 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:00.035222 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2\": container with ID starting with 613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2 not found: ID does not exist" containerID="613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2" Jul 07 12:45:00.035277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.035246 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2"} err="failed to get container status \"613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2\": rpc error: code = NotFound desc = could not find container \"613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2\": container with ID starting with 613dd67717826aeb81d70165184c62fafe08b03905898f908170e180894038d2 not found: ID does not exist" Jul 07 12:45:00.035277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.035261 2574 scope.go:117] "RemoveContainer" containerID="dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f" Jul 07 12:45:00.035442 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:00.035425 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f\": container with ID starting with dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f not found: ID does not exist" containerID="dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f" Jul 07 12:45:00.035497 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.035452 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f"} err="failed to get container status \"dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f\": rpc error: code = NotFound desc = could not find container \"dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f\": container with ID starting with dc9d25c6a5a52a5d7273e812f6a63c93368a8a5731c759cce0b7cb505dee6c4f not found: ID does not exist" Jul 07 12:45:00.035497 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.035466 2574 scope.go:117] "RemoveContainer" containerID="28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c" Jul 07 12:45:00.035663 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:00.035648 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c\": container with ID starting with 28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c not found: ID does not exist" containerID="28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c" Jul 07 12:45:00.035735 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.035664 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c"} err="failed to get container status \"28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c\": rpc error: code = NotFound desc = could not find container \"28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c\": container with ID starting with 28cccc679992de693fd6228787c82609605e8a66a039e51b5f2649642fb36e0c not found: ID does not exist" Jul 07 12:45:00.035735 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.035675 2574 scope.go:117] "RemoveContainer" containerID="d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a" Jul 07 12:45:00.035920 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:00.035901 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a\": container with ID starting with d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a not found: ID does not exist" containerID="d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a" Jul 07 12:45:00.035970 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.035925 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a"} err="failed to get container status \"d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a\": rpc error: code = NotFound desc = could not find container \"d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a\": container with ID starting with d726c23fb6115b343b949346d28a9e1425f40a690159d556657c06d7190ff46a not found: ID does not exist" Jul 07 12:45:00.035970 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.035942 2574 scope.go:117] "RemoveContainer" containerID="4c8d6fb9c33b045c55105d6b7832ba2b7a7d7a7899e4d8ec3756f59dd50eaa5b" Jul 07 12:45:00.036188 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:00.036170 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c8d6fb9c33b045c55105d6b7832ba2b7a7d7a7899e4d8ec3756f59dd50eaa5b\": container with ID starting with 4c8d6fb9c33b045c55105d6b7832ba2b7a7d7a7899e4d8ec3756f59dd50eaa5b not found: ID does not exist" containerID="4c8d6fb9c33b045c55105d6b7832ba2b7a7d7a7899e4d8ec3756f59dd50eaa5b" Jul 07 12:45:00.036243 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.036196 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c8d6fb9c33b045c55105d6b7832ba2b7a7d7a7899e4d8ec3756f59dd50eaa5b"} err="failed to get container status \"4c8d6fb9c33b045c55105d6b7832ba2b7a7d7a7899e4d8ec3756f59dd50eaa5b\": rpc error: code = NotFound desc = could not find container \"4c8d6fb9c33b045c55105d6b7832ba2b7a7d7a7899e4d8ec3756f59dd50eaa5b\": container with ID starting with 4c8d6fb9c33b045c55105d6b7832ba2b7a7d7a7899e4d8ec3756f59dd50eaa5b not found: ID does not exist" Jul 07 12:45:00.075280 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075251 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-main-tls\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.075280 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075280 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.075431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075298 2574 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23deede0-66e8-4510-87b3-e529f030053f-metrics-client-ca\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.075431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075311 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-config-out\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.075431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075325 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/23deede0-66e8-4510-87b3-e529f030053f-alertmanager-main-db\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.075431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075339 2574 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-config-volume\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.075431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075353 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.075431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075366 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-web-config\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.075431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075379 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.075431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075392 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk9mm\" (UniqueName: \"kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-kube-api-access-fk9mm\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.075431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075407 2574 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/23deede0-66e8-4510-87b3-e529f030053f-cluster-tls-config\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.075431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.075422 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/23deede0-66e8-4510-87b3-e529f030053f-tls-assets\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:00.303674 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.303587 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:45:00.311682 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.311633 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:45:00.342253 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342225 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:45:00.342572 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342552 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="prom-label-proxy" Jul 07 12:45:00.342629 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342574 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="prom-label-proxy" Jul 07 12:45:00.342629 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342588 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceaa562d-5f18-410c-b672-36a6384fcbb5" containerName="console" Jul 07 12:45:00.342629 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342593 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceaa562d-5f18-410c-b672-36a6384fcbb5" containerName="console" Jul 07 12:45:00.342629 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342601 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="config-reloader" Jul 07 12:45:00.342629 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342608 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="config-reloader" Jul 07 12:45:00.342629 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342619 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed4e2b39-79c5-4fbd-addc-2277488c6160" containerName="console" Jul 07 12:45:00.342629 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342624 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed4e2b39-79c5-4fbd-addc-2277488c6160" containerName="console" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342633 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="alertmanager" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342638 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="alertmanager" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342644 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy-web" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342648 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy-web" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342657 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="init-config-reloader" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342662 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="init-config-reloader" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342669 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy-metric" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342674 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy-metric" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342684 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342701 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342750 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceaa562d-5f18-410c-b672-36a6384fcbb5" containerName="console" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342756 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed4e2b39-79c5-4fbd-addc-2277488c6160" containerName="console" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342763 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy-web" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342771 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy-metric" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342777 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="prom-label-proxy" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342783 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="config-reloader" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342788 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="alertmanager" Jul 07 12:45:00.342879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.342793 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="23deede0-66e8-4510-87b3-e529f030053f" containerName="kube-rbac-proxy" Jul 07 12:45:00.346038 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.346021 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.349225 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.349186 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 12:45:00.350309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.349532 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 12:45:00.350309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.349618 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 12:45:00.350309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.349673 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 12:45:00.350519 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.349727 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 12:45:00.350519 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.349771 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 12:45:00.350519 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.349825 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 12:45:00.350519 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.349931 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-njdct\"" Jul 07 12:45:00.350519 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.349968 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 12:45:00.354633 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.354615 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 12:45:00.358938 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.358919 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:45:00.478179 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478143 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478179 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478184 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/634c42b9-63dd-404e-8e83-a330fb6d5b49-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478424 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478202 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/634c42b9-63dd-404e-8e83-a330fb6d5b49-tls-assets\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478424 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478224 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478424 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478311 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-web-config\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478424 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478363 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478424 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478389 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/634c42b9-63dd-404e-8e83-a330fb6d5b49-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478761 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478464 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478761 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478490 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478761 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478525 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/634c42b9-63dd-404e-8e83-a330fb6d5b49-config-out\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478761 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478547 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/634c42b9-63dd-404e-8e83-a330fb6d5b49-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478761 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skvjv\" (UniqueName: \"kubernetes.io/projected/634c42b9-63dd-404e-8e83-a330fb6d5b49-kube-api-access-skvjv\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.478761 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.478606 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-config-volume\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.579318 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.579209 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.579318 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.579264 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/634c42b9-63dd-404e-8e83-a330fb6d5b49-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.579318 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.579289 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/634c42b9-63dd-404e-8e83-a330fb6d5b49-tls-assets\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.579318 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.579313 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.579318 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.579333 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-web-config\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.579724 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.579359 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.579724 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.579378 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/634c42b9-63dd-404e-8e83-a330fb6d5b49-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.579724 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.579421 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.579724 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.579446 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.579724 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.579643 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/634c42b9-63dd-404e-8e83-a330fb6d5b49-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.580529 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.580499 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/634c42b9-63dd-404e-8e83-a330fb6d5b49-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.580640 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.580563 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/634c42b9-63dd-404e-8e83-a330fb6d5b49-config-out\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.580640 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.580596 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/634c42b9-63dd-404e-8e83-a330fb6d5b49-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.580640 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.580621 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skvjv\" (UniqueName: \"kubernetes.io/projected/634c42b9-63dd-404e-8e83-a330fb6d5b49-kube-api-access-skvjv\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.580829 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.580664 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-config-volume\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.582487 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.582461 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/634c42b9-63dd-404e-8e83-a330fb6d5b49-tls-assets\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.582645 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.582620 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.582748 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.582651 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.582815 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.582782 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.582916 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.582894 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-web-config\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.582970 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.582933 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.583082 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.583065 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/634c42b9-63dd-404e-8e83-a330fb6d5b49-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.583277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.583260 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.583277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.583268 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/634c42b9-63dd-404e-8e83-a330fb6d5b49-config-volume\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.584397 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.584380 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/634c42b9-63dd-404e-8e83-a330fb6d5b49-config-out\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.590792 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.590775 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skvjv\" (UniqueName: \"kubernetes.io/projected/634c42b9-63dd-404e-8e83-a330fb6d5b49-kube-api-access-skvjv\") pod \"alertmanager-main-0\" (UID: \"634c42b9-63dd-404e-8e83-a330fb6d5b49\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.658866 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.658837 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 12:45:00.791881 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.791852 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 12:45:00.793493 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:45:00.793470 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634c42b9_63dd_404e_8e83_a330fb6d5b49.slice/crio-0ab37a23154fa0dce32372ac6b006ba629e9dbb9ee7888aec09cb6b1e74ba37b WatchSource:0}: Error finding container 0ab37a23154fa0dce32372ac6b006ba629e9dbb9ee7888aec09cb6b1e74ba37b: Status 404 returned error can't find the container with id 0ab37a23154fa0dce32372ac6b006ba629e9dbb9ee7888aec09cb6b1e74ba37b Jul 07 12:45:00.982127 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.982088 2574 generic.go:356] "Generic (PLEG): container finished" podID="634c42b9-63dd-404e-8e83-a330fb6d5b49" containerID="41cd972fee908f3217f9e1a278e24132458d9367090b33306c36b433678683bb" exitCode=0 Jul 07 12:45:00.982585 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.982154 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"634c42b9-63dd-404e-8e83-a330fb6d5b49","Type":"ContainerDied","Data":"41cd972fee908f3217f9e1a278e24132458d9367090b33306c36b433678683bb"} Jul 07 12:45:00.982585 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:00.982229 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"634c42b9-63dd-404e-8e83-a330fb6d5b49","Type":"ContainerStarted","Data":"0ab37a23154fa0dce32372ac6b006ba629e9dbb9ee7888aec09cb6b1e74ba37b"} Jul 07 12:45:01.189650 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:01.189618 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23deede0-66e8-4510-87b3-e529f030053f" path="/var/lib/kubelet/pods/23deede0-66e8-4510-87b3-e529f030053f/volumes" Jul 07 12:45:01.989898 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:01.989865 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"634c42b9-63dd-404e-8e83-a330fb6d5b49","Type":"ContainerStarted","Data":"f6cf619c70768c58be0d16b5282c8c3f659d26b5f6bd5479979847b06af41f0f"} Jul 07 12:45:01.990392 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:01.989928 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"634c42b9-63dd-404e-8e83-a330fb6d5b49","Type":"ContainerStarted","Data":"c1029b5c97e3f8b83cf86aee880f97e99f79878b193ae81297410b7a2fab95c7"} Jul 07 12:45:01.990392 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:01.989957 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"634c42b9-63dd-404e-8e83-a330fb6d5b49","Type":"ContainerStarted","Data":"7c0604cca48bdf20456417b6aca90be09369ebe4825b2d12aacad50afbdd0d98"} Jul 07 12:45:01.990392 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:01.989980 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"634c42b9-63dd-404e-8e83-a330fb6d5b49","Type":"ContainerStarted","Data":"a399a9ae218914560887e86a7e7277201c14839e766023e5917e0d9eb9b875a7"} Jul 07 12:45:01.990392 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:01.990002 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"634c42b9-63dd-404e-8e83-a330fb6d5b49","Type":"ContainerStarted","Data":"e52182873ae0280463fc81b4cdb179c338a0ac8aaf9e9db473e7397d19994c33"} Jul 07 12:45:01.990392 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:01.990025 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"634c42b9-63dd-404e-8e83-a330fb6d5b49","Type":"ContainerStarted","Data":"1e8bdf6fc236712a24f233f3535a49b1d9d16bdb6e73955ec75abf9e7702caf7"} Jul 07 12:45:02.018869 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.018783 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.002878468 podStartE2EDuration="2.018761564s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.883096ms" isStatefulPod=false podCreationTimestamp="2026-07-07 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:45:02.016932792 +0000 UTC m=+149.460975990" watchObservedRunningTime="2026-07-07 12:45:02.018761564 +0000 UTC m=+149.462804745" Jul 07 12:45:02.602537 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.602504 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v"] Jul 07 12:45:02.605750 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.605727 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.608557 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.608535 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 12:45:02.608717 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.608678 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 12:45:02.608808 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.608722 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 12:45:02.608808 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.608636 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-dfls4\"" Jul 07 12:45:02.608808 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.608647 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 12:45:02.608808 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.608622 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 12:45:02.614280 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.614262 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 12:45:02.620913 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.620886 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v"] Jul 07 12:45:02.702786 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.702751 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/207bfd0e-99c9-4342-bf59-52261564614e-metrics-client-ca\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.702786 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.702787 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-telemeter-client-tls\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.703033 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.702809 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-secret-telemeter-client\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.703033 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.702833 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207bfd0e-99c9-4342-bf59-52261564614e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.703033 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.702894 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd8cm\" (UniqueName: \"kubernetes.io/projected/207bfd0e-99c9-4342-bf59-52261564614e-kube-api-access-kd8cm\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.703033 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.702933 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207bfd0e-99c9-4342-bf59-52261564614e-serving-certs-ca-bundle\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.703033 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.702969 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.703033 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.703012 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-federate-client-tls\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.804220 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.804185 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-federate-client-tls\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.804220 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.804226 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/207bfd0e-99c9-4342-bf59-52261564614e-metrics-client-ca\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.804485 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.804247 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-telemeter-client-tls\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.804485 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.804270 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-secret-telemeter-client\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.804485 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.804300 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207bfd0e-99c9-4342-bf59-52261564614e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.804485 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.804328 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd8cm\" (UniqueName: \"kubernetes.io/projected/207bfd0e-99c9-4342-bf59-52261564614e-kube-api-access-kd8cm\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.804485 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.804414 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207bfd0e-99c9-4342-bf59-52261564614e-serving-certs-ca-bundle\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.804755 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.804496 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.805084 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.805057 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/207bfd0e-99c9-4342-bf59-52261564614e-metrics-client-ca\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.805277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.805245 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207bfd0e-99c9-4342-bf59-52261564614e-serving-certs-ca-bundle\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.805474 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.805453 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/207bfd0e-99c9-4342-bf59-52261564614e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.806905 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.806885 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-federate-client-tls\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.807009 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.806990 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-telemeter-client-tls\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.807295 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.807277 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.807437 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.807418 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/207bfd0e-99c9-4342-bf59-52261564614e-secret-telemeter-client\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.816547 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.816517 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd8cm\" (UniqueName: \"kubernetes.io/projected/207bfd0e-99c9-4342-bf59-52261564614e-kube-api-access-kd8cm\") pod \"telemeter-client-6d5b88bb68-n8z2v\" (UID: \"207bfd0e-99c9-4342-bf59-52261564614e\") " pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:02.899221 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.899123 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:45:02.899586 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.899553 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154" gracePeriod=600 Jul 07 12:45:02.899985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.899958 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy-web" containerID="cri-o://bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b" gracePeriod=600 Jul 07 12:45:02.900093 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.900076 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy" containerID="cri-o://e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173" gracePeriod=600 Jul 07 12:45:02.900186 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.900163 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="prometheus" containerID="cri-o://f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba" gracePeriod=600 Jul 07 12:45:02.900246 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.900215 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="config-reloader" containerID="cri-o://e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737" gracePeriod=600 Jul 07 12:45:02.900286 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.900171 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="thanos-sidecar" containerID="cri-o://49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2" gracePeriod=600 Jul 07 12:45:02.916308 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:02.916241 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" Jul 07 12:45:03.078019 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.077936 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v"] Jul 07 12:45:03.080464 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:45:03.080432 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod207bfd0e_99c9_4342_bf59_52261564614e.slice/crio-dc4026936b65e566843d96b599df79e136566487f354a90253504644ab6674c3 WatchSource:0}: Error finding container dc4026936b65e566843d96b599df79e136566487f354a90253504644ab6674c3: Status 404 returned error can't find the container with id dc4026936b65e566843d96b599df79e136566487f354a90253504644ab6674c3 Jul 07 12:45:03.174626 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.174602 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:03.314448 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314418 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-2\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.314729 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314464 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-serving-certs-ca-bundle\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.314729 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314482 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-0\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.314729 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314512 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.314729 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314534 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-kubelet-serving-ca-bundle\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.314729 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314652 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-grpc-tls\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.314729 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314711 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-metrics-client-certs\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314737 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-metrics-client-ca\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314771 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-thanos-prometheus-http-client-file\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314807 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-1\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314855 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314864 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314925 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-db\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314936 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314970 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-tls-assets\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314971 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.314997 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-web-config\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.315023 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-trusted-ca-bundle\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315057 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.315053 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-config\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315682 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.315084 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-kube-rbac-proxy\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315682 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.315111 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw67m\" (UniqueName: \"kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-kube-api-access-qw67m\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315682 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.315139 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-config-out\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315682 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.315184 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-tls\") pod \"5177becb-9382-4534-b46e-50526b9a519c\" (UID: \"5177becb-9382-4534-b46e-50526b9a519c\") " Jul 07 12:45:03.315682 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.315466 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.315682 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.315485 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.315682 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.315499 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.315682 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.315614 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:45:03.316186 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.316037 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:45:03.317281 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.317210 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:45:03.317434 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.317347 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:45:03.318351 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.317822 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:45:03.318351 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.317819 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:45:03.318351 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.317905 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:45:03.318351 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.318013 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:45:03.318351 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.318271 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:45:03.318351 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.318283 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:45:03.318764 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.318606 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-config" (OuterVolumeSpecName: "config") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:45:03.318982 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.318956 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:45:03.319431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.319404 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-kube-api-access-qw67m" (OuterVolumeSpecName: "kube-api-access-qw67m") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "kube-api-access-qw67m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:45:03.319999 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.319975 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-config-out" (OuterVolumeSpecName: "config-out") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:45:03.320148 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.320129 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:45:03.320193 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.320152 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:45:03.331043 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.331015 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-web-config" (OuterVolumeSpecName: "web-config") pod "5177becb-9382-4534-b46e-50526b9a519c" (UID: "5177becb-9382-4534-b46e-50526b9a519c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:45:03.416271 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416189 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-tls-assets\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416271 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416219 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-web-config\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416271 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416229 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416271 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416241 2574 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-config\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416271 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416250 2574 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-kube-rbac-proxy\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416271 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416259 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw67m\" (UniqueName: \"kubernetes.io/projected/5177becb-9382-4534-b46e-50526b9a519c-kube-api-access-qw67m\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416271 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416266 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-config-out\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416271 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416274 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416618 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416282 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416618 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416292 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416618 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416301 2574 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-grpc-tls\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416618 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416309 2574 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-metrics-client-certs\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416618 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416317 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-configmap-metrics-client-ca\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416618 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416325 2574 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416618 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416333 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416618 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416341 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5177becb-9382-4534-b46e-50526b9a519c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:03.416618 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:03.416349 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5177becb-9382-4534-b46e-50526b9a519c-prometheus-k8s-db\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:45:04.000514 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.000471 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" event={"ID":"207bfd0e-99c9-4342-bf59-52261564614e","Type":"ContainerStarted","Data":"dc4026936b65e566843d96b599df79e136566487f354a90253504644ab6674c3"} Jul 07 12:45:04.003437 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003409 2574 generic.go:356] "Generic (PLEG): container finished" podID="5177becb-9382-4534-b46e-50526b9a519c" containerID="3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154" exitCode=0 Jul 07 12:45:04.003437 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003436 2574 generic.go:356] "Generic (PLEG): container finished" podID="5177becb-9382-4534-b46e-50526b9a519c" containerID="e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173" exitCode=0 Jul 07 12:45:04.003593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003443 2574 generic.go:356] "Generic (PLEG): container finished" podID="5177becb-9382-4534-b46e-50526b9a519c" containerID="bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b" exitCode=0 Jul 07 12:45:04.003593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003449 2574 generic.go:356] "Generic (PLEG): container finished" podID="5177becb-9382-4534-b46e-50526b9a519c" containerID="49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2" exitCode=0 Jul 07 12:45:04.003593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003454 2574 generic.go:356] "Generic (PLEG): container finished" podID="5177becb-9382-4534-b46e-50526b9a519c" containerID="e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737" exitCode=0 Jul 07 12:45:04.003593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003459 2574 generic.go:356] "Generic (PLEG): container finished" podID="5177becb-9382-4534-b46e-50526b9a519c" containerID="f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba" exitCode=0 Jul 07 12:45:04.003593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003480 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerDied","Data":"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154"} Jul 07 12:45:04.003593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003491 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.003593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003535 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerDied","Data":"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173"} Jul 07 12:45:04.003593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003564 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerDied","Data":"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b"} Jul 07 12:45:04.003593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003590 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerDied","Data":"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2"} Jul 07 12:45:04.003903 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003629 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerDied","Data":"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737"} Jul 07 12:45:04.003903 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003611 2574 scope.go:117] "RemoveContainer" containerID="3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154" Jul 07 12:45:04.003903 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003669 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerDied","Data":"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba"} Jul 07 12:45:04.003903 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.003724 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5177becb-9382-4534-b46e-50526b9a519c","Type":"ContainerDied","Data":"4d2df64f376b1ea7529aa3572396a7c593dda24d70ce795d38aebf8ddbedb37c"} Jul 07 12:45:04.011403 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.011385 2574 scope.go:117] "RemoveContainer" containerID="e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173" Jul 07 12:45:04.018626 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.018609 2574 scope.go:117] "RemoveContainer" containerID="bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b" Jul 07 12:45:04.025891 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.025873 2574 scope.go:117] "RemoveContainer" containerID="49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2" Jul 07 12:45:04.032491 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.032473 2574 scope.go:117] "RemoveContainer" containerID="e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737" Jul 07 12:45:04.036941 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.036922 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:45:04.041089 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.041073 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:45:04.041136 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.041104 2574 scope.go:117] "RemoveContainer" containerID="f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba" Jul 07 12:45:04.049876 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.049858 2574 scope.go:117] "RemoveContainer" containerID="319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597" Jul 07 12:45:04.057120 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.057101 2574 scope.go:117] "RemoveContainer" containerID="3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154" Jul 07 12:45:04.057379 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:04.057356 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": container with ID starting with 3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154 not found: ID does not exist" containerID="3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154" Jul 07 12:45:04.057477 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.057385 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154"} err="failed to get container status \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": rpc error: code = NotFound desc = could not find container \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": container with ID starting with 3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154 not found: ID does not exist" Jul 07 12:45:04.057477 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.057413 2574 scope.go:117] "RemoveContainer" containerID="e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173" Jul 07 12:45:04.057672 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:04.057657 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": container with ID starting with e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173 not found: ID does not exist" containerID="e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173" Jul 07 12:45:04.057737 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.057675 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173"} err="failed to get container status \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": rpc error: code = NotFound desc = could not find container \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": container with ID starting with e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173 not found: ID does not exist" Jul 07 12:45:04.057737 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.057708 2574 scope.go:117] "RemoveContainer" containerID="bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b" Jul 07 12:45:04.057937 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:04.057918 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": container with ID starting with bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b not found: ID does not exist" containerID="bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b" Jul 07 12:45:04.057980 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.057942 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b"} err="failed to get container status \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": rpc error: code = NotFound desc = could not find container \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": container with ID starting with bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b not found: ID does not exist" Jul 07 12:45:04.057980 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.057957 2574 scope.go:117] "RemoveContainer" containerID="49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2" Jul 07 12:45:04.058169 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:04.058154 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": container with ID starting with 49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2 not found: ID does not exist" containerID="49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2" Jul 07 12:45:04.058210 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.058172 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2"} err="failed to get container status \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": rpc error: code = NotFound desc = could not find container \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": container with ID starting with 49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2 not found: ID does not exist" Jul 07 12:45:04.058210 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.058185 2574 scope.go:117] "RemoveContainer" containerID="e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737" Jul 07 12:45:04.058407 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:04.058393 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": container with ID starting with e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737 not found: ID does not exist" containerID="e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737" Jul 07 12:45:04.058454 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.058409 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737"} err="failed to get container status \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": rpc error: code = NotFound desc = could not find container \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": container with ID starting with e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737 not found: ID does not exist" Jul 07 12:45:04.058454 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.058420 2574 scope.go:117] "RemoveContainer" containerID="f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba" Jul 07 12:45:04.058613 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:04.058599 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": container with ID starting with f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba not found: ID does not exist" containerID="f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba" Jul 07 12:45:04.058654 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.058615 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba"} err="failed to get container status \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": rpc error: code = NotFound desc = could not find container \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": container with ID starting with f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba not found: ID does not exist" Jul 07 12:45:04.058654 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.058627 2574 scope.go:117] "RemoveContainer" containerID="319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597" Jul 07 12:45:04.058885 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:45:04.058868 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": container with ID starting with 319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597 not found: ID does not exist" containerID="319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597" Jul 07 12:45:04.058926 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.058888 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597"} err="failed to get container status \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": rpc error: code = NotFound desc = could not find container \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": container with ID starting with 319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597 not found: ID does not exist" Jul 07 12:45:04.058926 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.058903 2574 scope.go:117] "RemoveContainer" containerID="3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154" Jul 07 12:45:04.059123 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.059108 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154"} err="failed to get container status \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": rpc error: code = NotFound desc = could not find container \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": container with ID starting with 3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154 not found: ID does not exist" Jul 07 12:45:04.059178 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.059124 2574 scope.go:117] "RemoveContainer" containerID="e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173" Jul 07 12:45:04.059327 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.059310 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173"} err="failed to get container status \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": rpc error: code = NotFound desc = could not find container \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": container with ID starting with e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173 not found: ID does not exist" Jul 07 12:45:04.059368 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.059327 2574 scope.go:117] "RemoveContainer" containerID="bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b" Jul 07 12:45:04.059522 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.059501 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b"} err="failed to get container status \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": rpc error: code = NotFound desc = could not find container \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": container with ID starting with bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b not found: ID does not exist" Jul 07 12:45:04.059568 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.059522 2574 scope.go:117] "RemoveContainer" containerID="49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2" Jul 07 12:45:04.059806 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.059787 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2"} err="failed to get container status \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": rpc error: code = NotFound desc = could not find container \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": container with ID starting with 49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2 not found: ID does not exist" Jul 07 12:45:04.059858 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.059807 2574 scope.go:117] "RemoveContainer" containerID="e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737" Jul 07 12:45:04.059999 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.059983 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737"} err="failed to get container status \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": rpc error: code = NotFound desc = could not find container \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": container with ID starting with e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737 not found: ID does not exist" Jul 07 12:45:04.060037 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.059999 2574 scope.go:117] "RemoveContainer" containerID="f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba" Jul 07 12:45:04.060176 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.060160 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba"} err="failed to get container status \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": rpc error: code = NotFound desc = could not find container \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": container with ID starting with f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba not found: ID does not exist" Jul 07 12:45:04.060216 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.060176 2574 scope.go:117] "RemoveContainer" containerID="319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597" Jul 07 12:45:04.060347 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.060331 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597"} err="failed to get container status \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": rpc error: code = NotFound desc = could not find container \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": container with ID starting with 319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597 not found: ID does not exist" Jul 07 12:45:04.060389 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.060347 2574 scope.go:117] "RemoveContainer" containerID="3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154" Jul 07 12:45:04.060588 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.060565 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154"} err="failed to get container status \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": rpc error: code = NotFound desc = could not find container \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": container with ID starting with 3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154 not found: ID does not exist" Jul 07 12:45:04.060588 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.060586 2574 scope.go:117] "RemoveContainer" containerID="e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173" Jul 07 12:45:04.060844 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.060825 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173"} err="failed to get container status \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": rpc error: code = NotFound desc = could not find container \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": container with ID starting with e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173 not found: ID does not exist" Jul 07 12:45:04.060890 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.060846 2574 scope.go:117] "RemoveContainer" containerID="bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b" Jul 07 12:45:04.061081 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.061061 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b"} err="failed to get container status \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": rpc error: code = NotFound desc = could not find container \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": container with ID starting with bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b not found: ID does not exist" Jul 07 12:45:04.061123 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.061082 2574 scope.go:117] "RemoveContainer" containerID="49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2" Jul 07 12:45:04.061328 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.061290 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2"} err="failed to get container status \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": rpc error: code = NotFound desc = could not find container \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": container with ID starting with 49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2 not found: ID does not exist" Jul 07 12:45:04.061328 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.061308 2574 scope.go:117] "RemoveContainer" containerID="e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737" Jul 07 12:45:04.061533 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.061516 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737"} err="failed to get container status \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": rpc error: code = NotFound desc = could not find container \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": container with ID starting with e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737 not found: ID does not exist" Jul 07 12:45:04.061578 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.061535 2574 scope.go:117] "RemoveContainer" containerID="f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba" Jul 07 12:45:04.061759 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.061719 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba"} err="failed to get container status \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": rpc error: code = NotFound desc = could not find container \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": container with ID starting with f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba not found: ID does not exist" Jul 07 12:45:04.061759 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.061735 2574 scope.go:117] "RemoveContainer" containerID="319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597" Jul 07 12:45:04.061953 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.061937 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597"} err="failed to get container status \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": rpc error: code = NotFound desc = could not find container \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": container with ID starting with 319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597 not found: ID does not exist" Jul 07 12:45:04.062094 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.061953 2574 scope.go:117] "RemoveContainer" containerID="3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154" Jul 07 12:45:04.062303 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.062278 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154"} err="failed to get container status \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": rpc error: code = NotFound desc = could not find container \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": container with ID starting with 3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154 not found: ID does not exist" Jul 07 12:45:04.062389 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.062305 2574 scope.go:117] "RemoveContainer" containerID="e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173" Jul 07 12:45:04.062618 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.062594 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173"} err="failed to get container status \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": rpc error: code = NotFound desc = could not find container \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": container with ID starting with e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173 not found: ID does not exist" Jul 07 12:45:04.062763 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.062619 2574 scope.go:117] "RemoveContainer" containerID="bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b" Jul 07 12:45:04.062909 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.062887 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b"} err="failed to get container status \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": rpc error: code = NotFound desc = could not find container \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": container with ID starting with bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b not found: ID does not exist" Jul 07 12:45:04.062909 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.062909 2574 scope.go:117] "RemoveContainer" containerID="49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2" Jul 07 12:45:04.063214 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.063185 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2"} err="failed to get container status \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": rpc error: code = NotFound desc = could not find container \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": container with ID starting with 49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2 not found: ID does not exist" Jul 07 12:45:04.063214 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.063214 2574 scope.go:117] "RemoveContainer" containerID="e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737" Jul 07 12:45:04.063476 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.063461 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737"} err="failed to get container status \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": rpc error: code = NotFound desc = could not find container \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": container with ID starting with e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737 not found: ID does not exist" Jul 07 12:45:04.063476 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.063476 2574 scope.go:117] "RemoveContainer" containerID="f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba" Jul 07 12:45:04.063741 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.063725 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba"} err="failed to get container status \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": rpc error: code = NotFound desc = could not find container \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": container with ID starting with f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba not found: ID does not exist" Jul 07 12:45:04.063741 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.063740 2574 scope.go:117] "RemoveContainer" containerID="319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597" Jul 07 12:45:04.063960 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.063945 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597"} err="failed to get container status \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": rpc error: code = NotFound desc = could not find container \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": container with ID starting with 319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597 not found: ID does not exist" Jul 07 12:45:04.063960 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.063959 2574 scope.go:117] "RemoveContainer" containerID="3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154" Jul 07 12:45:04.064220 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.064205 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154"} err="failed to get container status \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": rpc error: code = NotFound desc = could not find container \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": container with ID starting with 3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154 not found: ID does not exist" Jul 07 12:45:04.064274 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.064220 2574 scope.go:117] "RemoveContainer" containerID="e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173" Jul 07 12:45:04.064449 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.064420 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173"} err="failed to get container status \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": rpc error: code = NotFound desc = could not find container \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": container with ID starting with e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173 not found: ID does not exist" Jul 07 12:45:04.064449 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.064450 2574 scope.go:117] "RemoveContainer" containerID="bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b" Jul 07 12:45:04.064677 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.064659 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:45:04.064753 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.064672 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b"} err="failed to get container status \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": rpc error: code = NotFound desc = could not find container \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": container with ID starting with bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b not found: ID does not exist" Jul 07 12:45:04.064753 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.064701 2574 scope.go:117] "RemoveContainer" containerID="49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2" Jul 07 12:45:04.064933 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.064914 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2"} err="failed to get container status \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": rpc error: code = NotFound desc = could not find container \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": container with ID starting with 49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2 not found: ID does not exist" Jul 07 12:45:04.064978 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.064933 2574 scope.go:117] "RemoveContainer" containerID="e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737" Jul 07 12:45:04.065083 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065065 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="init-config-reloader" Jul 07 12:45:04.065145 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065088 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="init-config-reloader" Jul 07 12:45:04.065145 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065101 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="config-reloader" Jul 07 12:45:04.065145 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065108 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="config-reloader" Jul 07 12:45:04.065145 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065114 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy" Jul 07 12:45:04.065145 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065119 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy" Jul 07 12:45:04.065145 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065131 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy-thanos" Jul 07 12:45:04.065145 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065139 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy-thanos" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065151 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="thanos-sidecar" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065158 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="thanos-sidecar" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065175 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="prometheus" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065181 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="prometheus" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065191 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy-web" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065197 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy-web" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065112 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737"} err="failed to get container status \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": rpc error: code = NotFound desc = could not find container \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": container with ID starting with e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737 not found: ID does not exist" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065245 2574 scope.go:117] "RemoveContainer" containerID="f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065269 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="config-reloader" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065281 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="prometheus" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065291 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065300 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="thanos-sidecar" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065312 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy-web" Jul 07 12:45:04.065352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065321 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5177becb-9382-4534-b46e-50526b9a519c" containerName="kube-rbac-proxy-thanos" Jul 07 12:45:04.065854 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065479 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba"} err="failed to get container status \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": rpc error: code = NotFound desc = could not find container \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": container with ID starting with f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba not found: ID does not exist" Jul 07 12:45:04.065854 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065500 2574 scope.go:117] "RemoveContainer" containerID="319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597" Jul 07 12:45:04.065854 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065715 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597"} err="failed to get container status \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": rpc error: code = NotFound desc = could not find container \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": container with ID starting with 319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597 not found: ID does not exist" Jul 07 12:45:04.065854 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065728 2574 scope.go:117] "RemoveContainer" containerID="3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154" Jul 07 12:45:04.066050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065922 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154"} err="failed to get container status \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": rpc error: code = NotFound desc = could not find container \"3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154\": container with ID starting with 3bad06b58c3d2369b46e35bd23c540703043d58d1766c4f587288da519c24154 not found: ID does not exist" Jul 07 12:45:04.066050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.065945 2574 scope.go:117] "RemoveContainer" containerID="e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173" Jul 07 12:45:04.066206 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.066187 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173"} err="failed to get container status \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": rpc error: code = NotFound desc = could not find container \"e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173\": container with ID starting with e1bd94eb17f88dc8e0c2a3d85da6f20290c09840847f6bd1da9a6b3022874173 not found: ID does not exist" Jul 07 12:45:04.066248 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.066207 2574 scope.go:117] "RemoveContainer" containerID="bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b" Jul 07 12:45:04.066470 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.066446 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b"} err="failed to get container status \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": rpc error: code = NotFound desc = could not find container \"bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b\": container with ID starting with bbc189d559b86a44db9748f5815b5fbc768dc927dcf53fc962f945bbaaf13e9b not found: ID does not exist" Jul 07 12:45:04.066523 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.066473 2574 scope.go:117] "RemoveContainer" containerID="49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2" Jul 07 12:45:04.066712 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.066673 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2"} err="failed to get container status \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": rpc error: code = NotFound desc = could not find container \"49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2\": container with ID starting with 49faf239ba5c46a8f123a269640a24f12303a8c9b10590568c7ce4e3609cdef2 not found: ID does not exist" Jul 07 12:45:04.066712 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.066710 2574 scope.go:117] "RemoveContainer" containerID="e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737" Jul 07 12:45:04.066919 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.066900 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737"} err="failed to get container status \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": rpc error: code = NotFound desc = could not find container \"e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737\": container with ID starting with e43969d84362446bca107b6e91bc83d367112ebe0bfb1c1ef15bec02388ab737 not found: ID does not exist" Jul 07 12:45:04.066963 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.066920 2574 scope.go:117] "RemoveContainer" containerID="f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba" Jul 07 12:45:04.067106 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.067091 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba"} err="failed to get container status \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": rpc error: code = NotFound desc = could not find container \"f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba\": container with ID starting with f86e63497cd282ecfd327406f0b98566064363c817c659fc90723497d71698ba not found: ID does not exist" Jul 07 12:45:04.067161 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.067106 2574 scope.go:117] "RemoveContainer" containerID="319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597" Jul 07 12:45:04.067306 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.067290 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597"} err="failed to get container status \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": rpc error: code = NotFound desc = could not find container \"319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597\": container with ID starting with 319d3b7f534d25dd7cd83c717af89dbae46cc784dd99cd0c8a6037c1a131e597 not found: ID does not exist" Jul 07 12:45:04.070014 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.069996 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.072661 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.072644 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 12:45:04.072777 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.072672 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 12:45:04.072777 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.072718 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 12:45:04.073009 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.072988 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 12:45:04.073171 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.073131 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 12:45:04.073307 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.073196 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 12:45:04.073307 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.073252 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 12:45:04.073307 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.073273 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d5ij900o8bqe6\"" Jul 07 12:45:04.073516 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.073320 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 12:45:04.073516 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.073434 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 12:45:04.073646 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.073632 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 12:45:04.073733 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.073655 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nvbjk\"" Jul 07 12:45:04.073890 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.073849 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 12:45:04.074020 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.073960 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 12:45:04.078098 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.078053 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 12:45:04.079491 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.079474 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 12:45:04.087500 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.087479 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:45:04.222857 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.222819 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.222857 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.222861 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-config\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.222881 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.222898 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.222918 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.222936 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7slp\" (UniqueName: \"kubernetes.io/projected/d968574c-da0c-4d18-adc4-9d76d99a8035-kube-api-access-d7slp\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.222953 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.222981 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223021 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223061 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d968574c-da0c-4d18-adc4-9d76d99a8035-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223302 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223081 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223302 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223106 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223302 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223129 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223302 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223178 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d968574c-da0c-4d18-adc4-9d76d99a8035-config-out\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223302 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223206 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223302 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223238 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223302 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223259 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223302 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223276 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223530 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223317 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.223530 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.223341 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-web-config\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324231 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324129 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324231 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324194 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324231 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324225 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324514 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324263 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d968574c-da0c-4d18-adc4-9d76d99a8035-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324514 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324285 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324514 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324466 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324768 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324518 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324768 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324559 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d968574c-da0c-4d18-adc4-9d76d99a8035-config-out\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324768 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324586 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324768 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324594 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324768 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324653 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324768 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324707 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.324768 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324739 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.325124 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324774 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.325124 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324810 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-web-config\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.325124 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324846 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.325124 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324873 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-config\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.325124 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324899 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.325124 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324926 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.325124 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.324972 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.325124 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.325005 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7slp\" (UniqueName: \"kubernetes.io/projected/d968574c-da0c-4d18-adc4-9d76d99a8035-kube-api-access-d7slp\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.325529 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.325249 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.325529 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.325443 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.326050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.326025 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.327863 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.327538 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d968574c-da0c-4d18-adc4-9d76d99a8035-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.327863 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.327642 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.328191 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.328134 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-web-config\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.330615 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.328875 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-config\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.330615 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.328903 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.330615 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.329484 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.330615 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.329496 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.330615 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.330117 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.330615 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.330542 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.330615 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.330571 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d968574c-da0c-4d18-adc4-9d76d99a8035-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.331441 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.331395 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.331548 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.331452 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d968574c-da0c-4d18-adc4-9d76d99a8035-config-out\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.332050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.332005 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.332369 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.332339 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.332621 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.332597 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d968574c-da0c-4d18-adc4-9d76d99a8035-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.334328 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.334306 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7slp\" (UniqueName: \"kubernetes.io/projected/d968574c-da0c-4d18-adc4-9d76d99a8035-kube-api-access-d7slp\") pod \"prometheus-k8s-0\" (UID: \"d968574c-da0c-4d18-adc4-9d76d99a8035\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.381340 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.381305 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:04.532546 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:04.532520 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 12:45:04.534758 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:45:04.534731 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd968574c_da0c_4d18_adc4_9d76d99a8035.slice/crio-505ddf9e25df74a3d7e00f0dac219f3c8c9de914c66f751e94981ff38892f0a9 WatchSource:0}: Error finding container 505ddf9e25df74a3d7e00f0dac219f3c8c9de914c66f751e94981ff38892f0a9: Status 404 returned error can't find the container with id 505ddf9e25df74a3d7e00f0dac219f3c8c9de914c66f751e94981ff38892f0a9 Jul 07 12:45:05.009355 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:05.009313 2574 generic.go:356] "Generic (PLEG): container finished" podID="d968574c-da0c-4d18-adc4-9d76d99a8035" containerID="7fff11fa59f2def659dbe67ef63700c40804535c6d14c59ce353489b12713730" exitCode=0 Jul 07 12:45:05.009556 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:05.009400 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d968574c-da0c-4d18-adc4-9d76d99a8035","Type":"ContainerDied","Data":"7fff11fa59f2def659dbe67ef63700c40804535c6d14c59ce353489b12713730"} Jul 07 12:45:05.009556 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:05.009483 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d968574c-da0c-4d18-adc4-9d76d99a8035","Type":"ContainerStarted","Data":"505ddf9e25df74a3d7e00f0dac219f3c8c9de914c66f751e94981ff38892f0a9"} Jul 07 12:45:05.195641 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:05.195597 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5177becb-9382-4534-b46e-50526b9a519c" path="/var/lib/kubelet/pods/5177becb-9382-4534-b46e-50526b9a519c/volumes" Jul 07 12:45:06.017818 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:06.017717 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" event={"ID":"207bfd0e-99c9-4342-bf59-52261564614e","Type":"ContainerStarted","Data":"d96c3caa718d6544cf51d8088cefe51fca5fa8fd36d7fd26a0bab32433cd0c73"} Jul 07 12:45:06.017818 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:06.017787 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" event={"ID":"207bfd0e-99c9-4342-bf59-52261564614e","Type":"ContainerStarted","Data":"ac79ddb4bb68d12e3e1f7fc85fb231324a7ca409c9a203eff9d5045dc6dafbf7"} Jul 07 12:45:06.017818 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:06.017812 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" event={"ID":"207bfd0e-99c9-4342-bf59-52261564614e","Type":"ContainerStarted","Data":"105f27782b6df5fc4143f1afe00e8279b33aebb1eacbde67f8b352442136a749"} Jul 07 12:45:06.021004 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:06.020977 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d968574c-da0c-4d18-adc4-9d76d99a8035","Type":"ContainerStarted","Data":"891781ca71e76cabffcb7a082709fd45b21708558debc596e2c8dfa21825775d"} Jul 07 12:45:06.021141 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:06.021045 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d968574c-da0c-4d18-adc4-9d76d99a8035","Type":"ContainerStarted","Data":"5b8a60916024a0d7accbf25bfc0402993a72bd0072266327520b950d45ddc93d"} Jul 07 12:45:06.021141 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:06.021093 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d968574c-da0c-4d18-adc4-9d76d99a8035","Type":"ContainerStarted","Data":"80a614dbcb5d54692292a9f2f9de6d6517f7b1431fe57eaaf4c6083647fda1ec"} Jul 07 12:45:06.021141 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:06.021135 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d968574c-da0c-4d18-adc4-9d76d99a8035","Type":"ContainerStarted","Data":"5f8ac19fe7444bc20c57b8fcc29811086bddff703c0dc9137373ad2c6449564b"} Jul 07 12:45:06.021296 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:06.021182 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d968574c-da0c-4d18-adc4-9d76d99a8035","Type":"ContainerStarted","Data":"714f2744191167cd460c4b80c99a3f42da85b89cf87118a028cf1c238a0300f1"} Jul 07 12:45:06.021296 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:06.021223 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d968574c-da0c-4d18-adc4-9d76d99a8035","Type":"ContainerStarted","Data":"c549a4425e3f2f2f97737bff36e7a3c87b865cbf0cad96fccd4dfb94a6fe6263"} Jul 07 12:45:06.043006 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:06.042958 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6d5b88bb68-n8z2v" podStartSLOduration=1.6492631960000002 podStartE2EDuration="4.042940141s" totalImagesPullingTime="2.393676945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:45:06.040556123 +0000 UTC m=+153.484599326" watchObservedRunningTime="2026-07-07 12:45:06.042940141 +0000 UTC m=+153.486983320" Jul 07 12:45:06.071311 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:06.071259 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.05002144 podStartE2EDuration="2.071243408s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.221968ms" isStatefulPod=false podCreationTimestamp="2026-07-07 12:45:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:45:06.070047732 +0000 UTC m=+153.514090912" watchObservedRunningTime="2026-07-07 12:45:06.071243408 +0000 UTC m=+153.515286587" Jul 07 12:45:07.405149 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:07.405107 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:45:07.504954 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:07.504923 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:45:07.857952 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:07.857925 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:45:09.382364 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:09.382333 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:45:17.626986 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:17.626936 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:45:17.728638 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:17.728597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:45:17.790184 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:17.790126 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:45:49.053167 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:49.053125 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:45:51.725176 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:51.725140 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:45:51.826078 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:45:51.826043 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:04.382251 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:04.382215 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:46:04.397728 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:04.397673 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:46:05.015754 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:05.015709 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:05.276593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:05.276513 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 12:46:06.285583 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:06.285546 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:12.754983 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:12.754943 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:12.814538 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:12.814513 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:19.267933 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:19.267898 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:19.354126 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:19.354099 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:24.753670 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:24.753634 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:24.856583 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:24.856537 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:29.856006 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:29.855955 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:29.966008 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:29.965972 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:30.248164 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:30.248120 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:38.805160 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:38.805119 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:46:38.987960 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:46:38.987932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:16.573631 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:16.573534 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:17.652505 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:17.652468 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:33.002864 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:33.002837 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 12:47:35.239720 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:35.239668 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:35.352894 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:35.352850 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:38.011884 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:38.011850 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:39.376887 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:39.376851 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:41.650671 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:41.650638 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:41.759096 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:41.759059 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:44.010979 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.010946 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:44.378580 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.378503 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:44.563037 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.563007 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hqxnt"] Jul 07 12:47:44.566154 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.566135 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" Jul 07 12:47:44.577413 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.577380 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjwph\" (UniqueName: \"kubernetes.io/projected/18152d73-db9e-4dc4-9618-be961079e35b-kube-api-access-pjwph\") pod \"cert-manager-webhook-6486ffd997-hqxnt\" (UID: \"18152d73-db9e-4dc4-9618-be961079e35b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" Jul 07 12:47:44.577531 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.577425 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18152d73-db9e-4dc4-9618-be961079e35b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hqxnt\" (UID: \"18152d73-db9e-4dc4-9618-be961079e35b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" Jul 07 12:47:44.577531 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.577509 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8vw47\"" Jul 07 12:47:44.580234 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.580218 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 12:47:44.588235 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.588216 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 12:47:44.601137 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.601112 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hqxnt"] Jul 07 12:47:44.678366 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.678300 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18152d73-db9e-4dc4-9618-be961079e35b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hqxnt\" (UID: \"18152d73-db9e-4dc4-9618-be961079e35b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" Jul 07 12:47:44.678478 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.678403 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjwph\" (UniqueName: \"kubernetes.io/projected/18152d73-db9e-4dc4-9618-be961079e35b-kube-api-access-pjwph\") pod \"cert-manager-webhook-6486ffd997-hqxnt\" (UID: \"18152d73-db9e-4dc4-9618-be961079e35b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" Jul 07 12:47:44.694040 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.694018 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjwph\" (UniqueName: \"kubernetes.io/projected/18152d73-db9e-4dc4-9618-be961079e35b-kube-api-access-pjwph\") pod \"cert-manager-webhook-6486ffd997-hqxnt\" (UID: \"18152d73-db9e-4dc4-9618-be961079e35b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" Jul 07 12:47:44.704800 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.704776 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18152d73-db9e-4dc4-9618-be961079e35b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hqxnt\" (UID: \"18152d73-db9e-4dc4-9618-be961079e35b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" Jul 07 12:47:44.885918 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:44.885887 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" Jul 07 12:47:45.024745 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:45.024599 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hqxnt"] Jul 07 12:47:45.027583 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:47:45.027555 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18152d73_db9e_4dc4_9618_be961079e35b.slice/crio-808cb219b404961a5ffba129383123324fd8a471e5d3c4aca77067a8910823ea WatchSource:0}: Error finding container 808cb219b404961a5ffba129383123324fd8a471e5d3c4aca77067a8910823ea: Status 404 returned error can't find the container with id 808cb219b404961a5ffba129383123324fd8a471e5d3c4aca77067a8910823ea Jul 07 12:47:45.029460 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:45.029443 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 12:47:45.653816 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:45.653775 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" event={"ID":"18152d73-db9e-4dc4-9618-be961079e35b","Type":"ContainerStarted","Data":"808cb219b404961a5ffba129383123324fd8a471e5d3c4aca77067a8910823ea"} Jul 07 12:47:46.232398 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:46.232364 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:46.710122 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:46.710083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:48.665286 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:48.665255 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:48.668631 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:48.668600 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" event={"ID":"18152d73-db9e-4dc4-9618-be961079e35b","Type":"ContainerStarted","Data":"8168f2dc46f3cf38d4656b568202cbeefbf943c4a3b2965507976912177539b2"} Jul 07 12:47:48.668793 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:48.668642 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" Jul 07 12:47:48.702997 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:48.702953 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" podStartSLOduration=1.866391495 podStartE2EDuration="4.702939113s" totalImagesPullingTime="2.836547618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:47:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:47:48.702913872 +0000 UTC m=+316.146957052" watchObservedRunningTime="2026-07-07 12:47:48.702939113 +0000 UTC m=+316.146982293" Jul 07 12:47:48.852338 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:48.852303 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:51.911566 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:51.911531 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-llrjt"] Jul 07 12:47:51.914937 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:51.914921 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-llrjt" Jul 07 12:47:51.917998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:51.917979 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dm6t4\"" Jul 07 12:47:51.935288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:51.935269 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-llrjt"] Jul 07 12:47:51.938010 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:51.937992 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21f6695f-b2fa-4e5c-807d-13e786dedf83-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-llrjt\" (UID: \"21f6695f-b2fa-4e5c-807d-13e786dedf83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-llrjt" Jul 07 12:47:51.938102 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:51.938041 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwmx2\" (UniqueName: \"kubernetes.io/projected/21f6695f-b2fa-4e5c-807d-13e786dedf83-kube-api-access-lwmx2\") pod \"cert-manager-cainjector-559cdd8c87-llrjt\" (UID: \"21f6695f-b2fa-4e5c-807d-13e786dedf83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-llrjt" Jul 07 12:47:52.039200 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:52.039174 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwmx2\" (UniqueName: \"kubernetes.io/projected/21f6695f-b2fa-4e5c-807d-13e786dedf83-kube-api-access-lwmx2\") pod \"cert-manager-cainjector-559cdd8c87-llrjt\" (UID: \"21f6695f-b2fa-4e5c-807d-13e786dedf83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-llrjt" Jul 07 12:47:52.039310 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:52.039261 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21f6695f-b2fa-4e5c-807d-13e786dedf83-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-llrjt\" (UID: \"21f6695f-b2fa-4e5c-807d-13e786dedf83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-llrjt" Jul 07 12:47:52.049743 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:52.049725 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwmx2\" (UniqueName: \"kubernetes.io/projected/21f6695f-b2fa-4e5c-807d-13e786dedf83-kube-api-access-lwmx2\") pod \"cert-manager-cainjector-559cdd8c87-llrjt\" (UID: \"21f6695f-b2fa-4e5c-807d-13e786dedf83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-llrjt" Jul 07 12:47:52.050332 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:52.050305 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21f6695f-b2fa-4e5c-807d-13e786dedf83-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-llrjt\" (UID: \"21f6695f-b2fa-4e5c-807d-13e786dedf83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-llrjt" Jul 07 12:47:52.224178 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:52.224149 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-llrjt" Jul 07 12:47:52.355564 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:52.355540 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-llrjt"] Jul 07 12:47:52.358056 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:47:52.358029 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21f6695f_b2fa_4e5c_807d_13e786dedf83.slice/crio-e66e5f9eb03e3b95013ceccb454fab63b6ba251fe099fd1cd8ae3443f574a51d WatchSource:0}: Error finding container e66e5f9eb03e3b95013ceccb454fab63b6ba251fe099fd1cd8ae3443f574a51d: Status 404 returned error can't find the container with id e66e5f9eb03e3b95013ceccb454fab63b6ba251fe099fd1cd8ae3443f574a51d Jul 07 12:47:52.686580 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:52.686502 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-llrjt" event={"ID":"21f6695f-b2fa-4e5c-807d-13e786dedf83","Type":"ContainerStarted","Data":"af32949c8baad5af75688f793e60229f4ebad2626e675ae0e4e1c105395ac9d9"} Jul 07 12:47:52.686580 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:52.686543 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-llrjt" event={"ID":"21f6695f-b2fa-4e5c-807d-13e786dedf83","Type":"ContainerStarted","Data":"e66e5f9eb03e3b95013ceccb454fab63b6ba251fe099fd1cd8ae3443f574a51d"} Jul 07 12:47:52.717062 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:52.717013 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-llrjt" podStartSLOduration=1.7169973459999999 podStartE2EDuration="1.716997346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:47:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:47:52.715127394 +0000 UTC m=+320.159170601" watchObservedRunningTime="2026-07-07 12:47:52.716997346 +0000 UTC m=+320.161040524" Jul 07 12:47:53.209997 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:53.209965 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:53.346226 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:53.346197 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:54.675081 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:54.675049 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hqxnt" Jul 07 12:47:54.763556 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:54.763528 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:55.637431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:55.637393 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:47:55.749863 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:47:55.749833 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:01.832649 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:01.832617 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:01.950071 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:01.950039 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:03.483902 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:03.483869 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:04.732734 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:04.732685 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wwflr"] Jul 07 12:48:04.736232 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:04.736212 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wwflr" Jul 07 12:48:04.739867 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:04.739850 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-bhz2p\"" Jul 07 12:48:04.752486 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:04.752461 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wwflr"] Jul 07 12:48:04.840363 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:04.840333 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bde3f726-6509-4b6f-9705-a3d0492e5b53-bound-sa-token\") pod \"cert-manager-55b4d954d5-wwflr\" (UID: \"bde3f726-6509-4b6f-9705-a3d0492e5b53\") " pod="cert-manager/cert-manager-55b4d954d5-wwflr" Jul 07 12:48:04.840538 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:04.840399 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbwcj\" (UniqueName: \"kubernetes.io/projected/bde3f726-6509-4b6f-9705-a3d0492e5b53-kube-api-access-nbwcj\") pod \"cert-manager-55b4d954d5-wwflr\" (UID: \"bde3f726-6509-4b6f-9705-a3d0492e5b53\") " pod="cert-manager/cert-manager-55b4d954d5-wwflr" Jul 07 12:48:04.941029 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:04.940985 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bde3f726-6509-4b6f-9705-a3d0492e5b53-bound-sa-token\") pod \"cert-manager-55b4d954d5-wwflr\" (UID: \"bde3f726-6509-4b6f-9705-a3d0492e5b53\") " pod="cert-manager/cert-manager-55b4d954d5-wwflr" Jul 07 12:48:04.941190 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:04.941084 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbwcj\" (UniqueName: \"kubernetes.io/projected/bde3f726-6509-4b6f-9705-a3d0492e5b53-kube-api-access-nbwcj\") pod \"cert-manager-55b4d954d5-wwflr\" (UID: \"bde3f726-6509-4b6f-9705-a3d0492e5b53\") " pod="cert-manager/cert-manager-55b4d954d5-wwflr" Jul 07 12:48:04.951665 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:04.951640 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bde3f726-6509-4b6f-9705-a3d0492e5b53-bound-sa-token\") pod \"cert-manager-55b4d954d5-wwflr\" (UID: \"bde3f726-6509-4b6f-9705-a3d0492e5b53\") " pod="cert-manager/cert-manager-55b4d954d5-wwflr" Jul 07 12:48:04.951817 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:04.951686 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbwcj\" (UniqueName: \"kubernetes.io/projected/bde3f726-6509-4b6f-9705-a3d0492e5b53-kube-api-access-nbwcj\") pod \"cert-manager-55b4d954d5-wwflr\" (UID: \"bde3f726-6509-4b6f-9705-a3d0492e5b53\") " pod="cert-manager/cert-manager-55b4d954d5-wwflr" Jul 07 12:48:05.045749 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:05.045663 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wwflr" Jul 07 12:48:05.168954 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:05.168926 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wwflr"] Jul 07 12:48:05.171431 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:48:05.171407 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbde3f726_6509_4b6f_9705_a3d0492e5b53.slice/crio-aecb6fc28af2024c54148c2a2bc12817fe051c1476c9cf2af54d8915ea9bd007 WatchSource:0}: Error finding container aecb6fc28af2024c54148c2a2bc12817fe051c1476c9cf2af54d8915ea9bd007: Status 404 returned error can't find the container with id aecb6fc28af2024c54148c2a2bc12817fe051c1476c9cf2af54d8915ea9bd007 Jul 07 12:48:05.743281 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:05.743247 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wwflr" event={"ID":"bde3f726-6509-4b6f-9705-a3d0492e5b53","Type":"ContainerStarted","Data":"0281fe7d6e126280db0d8fdf528c815a6595f56dc7152611115111265808a92c"} Jul 07 12:48:05.743281 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:05.743289 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wwflr" event={"ID":"bde3f726-6509-4b6f-9705-a3d0492e5b53","Type":"ContainerStarted","Data":"aecb6fc28af2024c54148c2a2bc12817fe051c1476c9cf2af54d8915ea9bd007"} Jul 07 12:48:05.773811 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:05.773756 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-wwflr" podStartSLOduration=1.773734212 podStartE2EDuration="1.773734212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:05.771903792 +0000 UTC m=+333.215946974" watchObservedRunningTime="2026-07-07 12:48:05.773734212 +0000 UTC m=+333.217777392" Jul 07 12:48:15.923622 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:15.923587 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:16.075225 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:16.075190 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:16.476218 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:16.476153 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:16.706964 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:16.706931 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:16.961926 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:16.961887 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:16.976253 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:16.976214 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss"] Jul 07 12:48:16.979654 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:16.979635 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:16.987721 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:16.987679 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 12:48:16.987840 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:16.987679 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 12:48:16.987840 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:16.987752 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 12:48:16.988179 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:16.988161 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 12:48:16.988261 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:16.988169 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7tjqs\"" Jul 07 12:48:17.045493 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.045455 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6662221e-c29d-40b4-a12b-b6724932a97b-manifests\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.045493 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.045497 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7p5l\" (UniqueName: \"kubernetes.io/projected/6662221e-c29d-40b4-a12b-b6724932a97b-kube-api-access-z7p5l\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.045742 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.045562 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6662221e-c29d-40b4-a12b-b6724932a97b-tmp\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.045742 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.045645 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6662221e-c29d-40b4-a12b-b6724932a97b-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.045846 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.045762 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6662221e-c29d-40b4-a12b-b6724932a97b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.146344 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.146307 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6662221e-c29d-40b4-a12b-b6724932a97b-manifests\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.146344 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.146346 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7p5l\" (UniqueName: \"kubernetes.io/projected/6662221e-c29d-40b4-a12b-b6724932a97b-kube-api-access-z7p5l\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.146601 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.146375 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6662221e-c29d-40b4-a12b-b6724932a97b-tmp\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.146601 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.146399 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6662221e-c29d-40b4-a12b-b6724932a97b-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.146601 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.146447 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6662221e-c29d-40b4-a12b-b6724932a97b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.146871 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.146847 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6662221e-c29d-40b4-a12b-b6724932a97b-manifests\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.149044 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.149023 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6662221e-c29d-40b4-a12b-b6724932a97b-tmp\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.149348 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.149326 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6662221e-c29d-40b4-a12b-b6724932a97b-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.149435 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.149370 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6662221e-c29d-40b4-a12b-b6724932a97b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.150636 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.150607 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss"] Jul 07 12:48:17.169841 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.169813 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7p5l\" (UniqueName: \"kubernetes.io/projected/6662221e-c29d-40b4-a12b-b6724932a97b-kube-api-access-z7p5l\") pod \"opendatahub-operator-controller-manager-68cfd86456-x4sss\" (UID: \"6662221e-c29d-40b4-a12b-b6724932a97b\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.264534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.264444 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:17.290653 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.290609 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:17.458734 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.458702 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss"] Jul 07 12:48:17.464105 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.464072 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:17.470615 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:48:17.470590 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6662221e_c29d_40b4_a12b_b6724932a97b.slice/crio-b0e764edc2b9c1cc0e4bd0066ec2ce2f5e1709c1a35d1b63faa24e55cdaafd40 WatchSource:0}: Error finding container b0e764edc2b9c1cc0e4bd0066ec2ce2f5e1709c1a35d1b63faa24e55cdaafd40: Status 404 returned error can't find the container with id b0e764edc2b9c1cc0e4bd0066ec2ce2f5e1709c1a35d1b63faa24e55cdaafd40 Jul 07 12:48:17.697711 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.697656 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:17.793119 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.793085 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" event={"ID":"6662221e-c29d-40b4-a12b-b6724932a97b","Type":"ContainerStarted","Data":"b0e764edc2b9c1cc0e4bd0066ec2ce2f5e1709c1a35d1b63faa24e55cdaafd40"} Jul 07 12:48:17.868041 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:17.868006 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:20.810126 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:20.810042 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" event={"ID":"6662221e-c29d-40b4-a12b-b6724932a97b","Type":"ContainerStarted","Data":"c94f725a4a6ae0f3de7751f8ae558febe6eb1f619deec3bb747c3f650cd2b084"} Jul 07 12:48:21.581815 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:21.581780 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:21.700773 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:21.700740 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:21.815582 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:21.815543 2574 generic.go:356] "Generic (PLEG): container finished" podID="6662221e-c29d-40b4-a12b-b6724932a97b" containerID="c94f725a4a6ae0f3de7751f8ae558febe6eb1f619deec3bb747c3f650cd2b084" exitCode=0 Jul 07 12:48:21.815582 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:21.815581 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" event={"ID":"6662221e-c29d-40b4-a12b-b6724932a97b","Type":"ContainerDied","Data":"c94f725a4a6ae0f3de7751f8ae558febe6eb1f619deec3bb747c3f650cd2b084"} Jul 07 12:48:22.822213 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:22.822173 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" event={"ID":"6662221e-c29d-40b4-a12b-b6724932a97b","Type":"ContainerStarted","Data":"b5080d7dfebbb7dee6a5865a019869be6ec5285014086cf5dcaca1f3127ac042"} Jul 07 12:48:22.822767 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:22.822312 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:22.859827 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:22.859770 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" podStartSLOduration=2.264727401 podStartE2EDuration="6.859750995s" totalImagesPullingTime="3.467381657s" totalInitContainerRuntime="1.127641937s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:22.858589354 +0000 UTC m=+350.302632532" watchObservedRunningTime="2026-07-07 12:48:22.859750995 +0000 UTC m=+350.303794173" Jul 07 12:48:30.101133 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:30.101024 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:30.504985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:30.504949 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:33.830031 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:33.829994 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-x4sss" Jul 07 12:48:36.778906 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:36.778866 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:37.038740 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.038620 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:37.568156 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.568121 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74684786d9-k2xkz"] Jul 07 12:48:37.576517 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.576492 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" Jul 07 12:48:37.579728 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.579708 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 12:48:37.580829 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.580812 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 12:48:37.580935 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.580871 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 12:48:37.580935 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.580885 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 12:48:37.580935 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.580886 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-n2k64\"" Jul 07 12:48:37.597828 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.597803 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74684786d9-k2xkz"] Jul 07 12:48:37.600086 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.600059 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:37.619878 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.619857 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66201680-5f59-4d9f-a032-f81913957842-tmp\") pod \"kube-auth-proxy-74684786d9-k2xkz\" (UID: \"66201680-5f59-4d9f-a032-f81913957842\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" Jul 07 12:48:37.620002 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.619885 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4nks\" (UniqueName: \"kubernetes.io/projected/66201680-5f59-4d9f-a032-f81913957842-kube-api-access-p4nks\") pod \"kube-auth-proxy-74684786d9-k2xkz\" (UID: \"66201680-5f59-4d9f-a032-f81913957842\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" Jul 07 12:48:37.620002 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.619975 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66201680-5f59-4d9f-a032-f81913957842-tls-certs\") pod \"kube-auth-proxy-74684786d9-k2xkz\" (UID: \"66201680-5f59-4d9f-a032-f81913957842\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" Jul 07 12:48:37.720802 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.720770 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4nks\" (UniqueName: \"kubernetes.io/projected/66201680-5f59-4d9f-a032-f81913957842-kube-api-access-p4nks\") pod \"kube-auth-proxy-74684786d9-k2xkz\" (UID: \"66201680-5f59-4d9f-a032-f81913957842\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" Jul 07 12:48:37.720964 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.720845 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66201680-5f59-4d9f-a032-f81913957842-tls-certs\") pod \"kube-auth-proxy-74684786d9-k2xkz\" (UID: \"66201680-5f59-4d9f-a032-f81913957842\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" Jul 07 12:48:37.721010 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.720998 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66201680-5f59-4d9f-a032-f81913957842-tmp\") pod \"kube-auth-proxy-74684786d9-k2xkz\" (UID: \"66201680-5f59-4d9f-a032-f81913957842\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" Jul 07 12:48:37.723069 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.723048 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66201680-5f59-4d9f-a032-f81913957842-tmp\") pod \"kube-auth-proxy-74684786d9-k2xkz\" (UID: \"66201680-5f59-4d9f-a032-f81913957842\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" Jul 07 12:48:37.723300 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.723279 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66201680-5f59-4d9f-a032-f81913957842-tls-certs\") pod \"kube-auth-proxy-74684786d9-k2xkz\" (UID: \"66201680-5f59-4d9f-a032-f81913957842\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" Jul 07 12:48:37.747917 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.747893 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4nks\" (UniqueName: \"kubernetes.io/projected/66201680-5f59-4d9f-a032-f81913957842-kube-api-access-p4nks\") pod \"kube-auth-proxy-74684786d9-k2xkz\" (UID: \"66201680-5f59-4d9f-a032-f81913957842\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" Jul 07 12:48:37.783634 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.783606 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:37.836721 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.836615 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ffz26"] Jul 07 12:48:37.841487 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.841456 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" Jul 07 12:48:37.846504 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.846483 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 07 12:48:37.849083 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.849062 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wc87k\"" Jul 07 12:48:37.885592 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.885569 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" Jul 07 12:48:37.922948 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.922913 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0624566-5357-4498-bc6b-92c36bb05ded-cert\") pod \"odh-model-controller-7998c67dc4-ffz26\" (UID: \"a0624566-5357-4498-bc6b-92c36bb05ded\") " pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" Jul 07 12:48:37.923094 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.923072 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffgtb\" (UniqueName: \"kubernetes.io/projected/a0624566-5357-4498-bc6b-92c36bb05ded-kube-api-access-ffgtb\") pod \"odh-model-controller-7998c67dc4-ffz26\" (UID: \"a0624566-5357-4498-bc6b-92c36bb05ded\") " pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" Jul 07 12:48:37.954562 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:37.954476 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ffz26"] Jul 07 12:48:38.023971 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.023933 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffgtb\" (UniqueName: \"kubernetes.io/projected/a0624566-5357-4498-bc6b-92c36bb05ded-kube-api-access-ffgtb\") pod \"odh-model-controller-7998c67dc4-ffz26\" (UID: \"a0624566-5357-4498-bc6b-92c36bb05ded\") " pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" Jul 07 12:48:38.024162 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.024024 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0624566-5357-4498-bc6b-92c36bb05ded-cert\") pod \"odh-model-controller-7998c67dc4-ffz26\" (UID: \"a0624566-5357-4498-bc6b-92c36bb05ded\") " pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" Jul 07 12:48:38.024225 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:48:38.024168 2574 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 07 12:48:38.024283 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:48:38.024232 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0624566-5357-4498-bc6b-92c36bb05ded-cert podName:a0624566-5357-4498-bc6b-92c36bb05ded nodeName:}" failed. No retries permitted until 2026-07-07 12:48:38.524210504 +0000 UTC m=+365.968253661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a0624566-5357-4498-bc6b-92c36bb05ded-cert") pod "odh-model-controller-7998c67dc4-ffz26" (UID: "a0624566-5357-4498-bc6b-92c36bb05ded") : secret "odh-model-controller-webhook-cert" not found Jul 07 12:48:38.056227 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:48:38.056199 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66201680_5f59_4d9f_a032_f81913957842.slice/crio-87aa2aa1b0daf6ce85b0e80a856fbec40644dcad48809004d6a7573a2bef5218 WatchSource:0}: Error finding container 87aa2aa1b0daf6ce85b0e80a856fbec40644dcad48809004d6a7573a2bef5218: Status 404 returned error can't find the container with id 87aa2aa1b0daf6ce85b0e80a856fbec40644dcad48809004d6a7573a2bef5218 Jul 07 12:48:38.059558 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.059530 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74684786d9-k2xkz"] Jul 07 12:48:38.061165 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.061145 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffgtb\" (UniqueName: \"kubernetes.io/projected/a0624566-5357-4498-bc6b-92c36bb05ded-kube-api-access-ffgtb\") pod \"odh-model-controller-7998c67dc4-ffz26\" (UID: \"a0624566-5357-4498-bc6b-92c36bb05ded\") " pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" Jul 07 12:48:38.079463 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.079443 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:38.349193 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.349153 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:38.527234 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.527195 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0624566-5357-4498-bc6b-92c36bb05ded-cert\") pod \"odh-model-controller-7998c67dc4-ffz26\" (UID: \"a0624566-5357-4498-bc6b-92c36bb05ded\") " pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" Jul 07 12:48:38.529657 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.529632 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0624566-5357-4498-bc6b-92c36bb05ded-cert\") pod \"odh-model-controller-7998c67dc4-ffz26\" (UID: \"a0624566-5357-4498-bc6b-92c36bb05ded\") " pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" Jul 07 12:48:38.595704 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.595672 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:38.753310 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.753271 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" Jul 07 12:48:38.899068 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.899029 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" event={"ID":"66201680-5f59-4d9f-a032-f81913957842","Type":"ContainerStarted","Data":"87aa2aa1b0daf6ce85b0e80a856fbec40644dcad48809004d6a7573a2bef5218"} Jul 07 12:48:38.919156 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:48:38.918474 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0624566_5357_4498_bc6b_92c36bb05ded.slice/crio-179712988c676475d6f1b4f71c7efeb66355b2bbe8ef4336a5a2249b62f72536 WatchSource:0}: Error finding container 179712988c676475d6f1b4f71c7efeb66355b2bbe8ef4336a5a2249b62f72536: Status 404 returned error can't find the container with id 179712988c676475d6f1b4f71c7efeb66355b2bbe8ef4336a5a2249b62f72536 Jul 07 12:48:38.921545 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:38.921282 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ffz26"] Jul 07 12:48:39.150805 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.150466 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:39.212050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.212004 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bzhg7"] Jul 07 12:48:39.215802 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.215777 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" Jul 07 12:48:39.222146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.222114 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-dqb9n\"" Jul 07 12:48:39.232252 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.232214 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 07 12:48:39.234614 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.234572 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8fhc\" (UniqueName: \"kubernetes.io/projected/5262fe54-c1bd-4ca2-bb12-46b8ba16254e-kube-api-access-j8fhc\") pod \"model-serving-api-54d89bd5fc-bzhg7\" (UID: \"5262fe54-c1bd-4ca2-bb12-46b8ba16254e\") " pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" Jul 07 12:48:39.234799 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.234652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5262fe54-c1bd-4ca2-bb12-46b8ba16254e-tls-certs\") pod \"model-serving-api-54d89bd5fc-bzhg7\" (UID: \"5262fe54-c1bd-4ca2-bb12-46b8ba16254e\") " pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" Jul 07 12:48:39.245455 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.245409 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bzhg7"] Jul 07 12:48:39.334675 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.334624 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:39.335849 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.335812 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8fhc\" (UniqueName: \"kubernetes.io/projected/5262fe54-c1bd-4ca2-bb12-46b8ba16254e-kube-api-access-j8fhc\") pod \"model-serving-api-54d89bd5fc-bzhg7\" (UID: \"5262fe54-c1bd-4ca2-bb12-46b8ba16254e\") " pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" Jul 07 12:48:39.336003 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.335870 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5262fe54-c1bd-4ca2-bb12-46b8ba16254e-tls-certs\") pod \"model-serving-api-54d89bd5fc-bzhg7\" (UID: \"5262fe54-c1bd-4ca2-bb12-46b8ba16254e\") " pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" Jul 07 12:48:39.338838 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.338813 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5262fe54-c1bd-4ca2-bb12-46b8ba16254e-tls-certs\") pod \"model-serving-api-54d89bd5fc-bzhg7\" (UID: \"5262fe54-c1bd-4ca2-bb12-46b8ba16254e\") " pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" Jul 07 12:48:39.355508 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.355476 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8fhc\" (UniqueName: \"kubernetes.io/projected/5262fe54-c1bd-4ca2-bb12-46b8ba16254e-kube-api-access-j8fhc\") pod \"model-serving-api-54d89bd5fc-bzhg7\" (UID: \"5262fe54-c1bd-4ca2-bb12-46b8ba16254e\") " pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" Jul 07 12:48:39.531315 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.531286 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" Jul 07 12:48:39.770941 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.770910 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bzhg7"] Jul 07 12:48:39.772456 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:48:39.772425 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5262fe54_c1bd_4ca2_bb12_46b8ba16254e.slice/crio-dc6279d285d4d33bd2d777cb9a99691ad7aac935a53e5dac1c0c925151e3d436 WatchSource:0}: Error finding container dc6279d285d4d33bd2d777cb9a99691ad7aac935a53e5dac1c0c925151e3d436: Status 404 returned error can't find the container with id dc6279d285d4d33bd2d777cb9a99691ad7aac935a53e5dac1c0c925151e3d436 Jul 07 12:48:39.847845 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.847726 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:39.876015 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.875959 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-68mm9"] Jul 07 12:48:39.879984 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.879955 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:39.883879 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.883849 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 12:48:39.884300 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.884279 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 12:48:39.884300 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.884294 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 12:48:39.884845 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.884540 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-964q8\"" Jul 07 12:48:39.884845 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.884587 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 12:48:39.913143 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.912619 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" event={"ID":"a0624566-5357-4498-bc6b-92c36bb05ded","Type":"ContainerStarted","Data":"179712988c676475d6f1b4f71c7efeb66355b2bbe8ef4336a5a2249b62f72536"} Jul 07 12:48:39.917540 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.917509 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" event={"ID":"5262fe54-c1bd-4ca2-bb12-46b8ba16254e","Type":"ContainerStarted","Data":"dc6279d285d4d33bd2d777cb9a99691ad7aac935a53e5dac1c0c925151e3d436"} Jul 07 12:48:39.931906 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.929829 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-68mm9"] Jul 07 12:48:39.941717 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.941670 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-cacerts\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:39.941953 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.941740 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:39.941953 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.941808 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8524c624-9c9f-4aaf-9848-d57ae754453d-local-certs\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:39.941953 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.941880 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:39.941953 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.941909 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:39.942220 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.942011 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxwxj\" (UniqueName: \"kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-kube-api-access-sxwxj\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:39.942220 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:39.942065 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-token\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.043024 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.042984 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-token\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.043204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.043048 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-cacerts\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.043204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.043082 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.043204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.043139 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8524c624-9c9f-4aaf-9848-d57ae754453d-local-certs\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.043204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.043177 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.043204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.043203 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.043514 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.043258 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxwxj\" (UniqueName: \"kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-kube-api-access-sxwxj\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.044518 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.044455 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.046971 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.046214 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.046971 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.046678 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8524c624-9c9f-4aaf-9848-d57ae754453d-local-certs\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.046971 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.046957 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-cacerts\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.047362 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.047085 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.085916 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.085885 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-token\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.086071 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.086042 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxwxj\" (UniqueName: \"kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-kube-api-access-sxwxj\") pod \"istiod-openshift-gateway-7574459656-68mm9\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:40.099348 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.099290 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:40.195096 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:40.195058 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:41.970527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:41.970434 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:42.055293 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:42.055262 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-68mm9"] Jul 07 12:48:42.275949 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:48:42.275911 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8524c624_9c9f_4aaf_9848_d57ae754453d.slice/crio-78273137a04a7173b78ea8e94c4256340a2091fce99133e83f0146a20e91fd63 WatchSource:0}: Error finding container 78273137a04a7173b78ea8e94c4256340a2091fce99133e83f0146a20e91fd63: Status 404 returned error can't find the container with id 78273137a04a7173b78ea8e94c4256340a2091fce99133e83f0146a20e91fd63 Jul 07 12:48:42.938197 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:42.938167 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" event={"ID":"8524c624-9c9f-4aaf-9848-d57ae754453d","Type":"ContainerStarted","Data":"78273137a04a7173b78ea8e94c4256340a2091fce99133e83f0146a20e91fd63"} Jul 07 12:48:43.946492 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:43.946451 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" event={"ID":"5262fe54-c1bd-4ca2-bb12-46b8ba16254e","Type":"ContainerStarted","Data":"da66bb130eb3a16e66569fc606156b44cd5159105dfacfcd6406581abc7ade72"} Jul 07 12:48:43.946972 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:43.946654 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" Jul 07 12:48:43.948289 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:43.948265 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" event={"ID":"a0624566-5357-4498-bc6b-92c36bb05ded","Type":"ContainerStarted","Data":"f02d8a29b4e0521ece1e63613101921b90de9623d2318a36d1e3411e1386ff45"} Jul 07 12:48:43.948416 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:43.948312 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" Jul 07 12:48:43.950093 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:43.950068 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" event={"ID":"66201680-5f59-4d9f-a032-f81913957842","Type":"ContainerStarted","Data":"aaab2422e1e954bb2f41c8bbc855e68e33a99dd6f3a3298efa0a7e25c4eed458"} Jul 07 12:48:43.974755 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:43.974701 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" podStartSLOduration=1.8399497729999998 podStartE2EDuration="4.974671458s" totalImagesPullingTime="3.134721685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:43.972576017 +0000 UTC m=+371.416619197" watchObservedRunningTime="2026-07-07 12:48:43.974671458 +0000 UTC m=+371.418714636" Jul 07 12:48:44.027536 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:44.027472 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74684786d9-k2xkz" podStartSLOduration=2.183507828 podStartE2EDuration="7.027449824s" totalImagesPullingTime="4.843941996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:43.995838161 +0000 UTC m=+371.439881335" watchObservedRunningTime="2026-07-07 12:48:44.027449824 +0000 UTC m=+371.471493004" Jul 07 12:48:44.028869 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:44.028824 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" podStartSLOduration=3.027927666 podStartE2EDuration="7.028812291s" totalImagesPullingTime="4.000884625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:44.025140898 +0000 UTC m=+371.469184115" watchObservedRunningTime="2026-07-07 12:48:44.028812291 +0000 UTC m=+371.472855473" Jul 07 12:48:44.862757 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:44.862715 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 12:48:44.862857 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:44.862785 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 12:48:44.957972 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:44.957928 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" event={"ID":"8524c624-9c9f-4aaf-9848-d57ae754453d","Type":"ContainerStarted","Data":"48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423"} Jul 07 12:48:44.958473 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:44.958055 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:44.999364 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:44.999313 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" podStartSLOduration=3.4150731309999998 podStartE2EDuration="5.999297251s" totalImagesPullingTime="2.58422412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:44.998678605 +0000 UTC m=+372.442721784" watchObservedRunningTime="2026-07-07 12:48:44.999297251 +0000 UTC m=+372.443340430" Jul 07 12:48:45.838834 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:45.838805 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:45.964994 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:45.964961 2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-68mm9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 12:48:45.965364 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:45.965015 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" podUID="8524c624-9c9f-4aaf-9848-d57ae754453d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 12:48:46.176037 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:46.175932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:48.254898 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:48.254849 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:48.643036 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:48.642952 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:48.963989 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:48.963957 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:49.041024 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:49.040992 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:54.960350 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:54.960316 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-ffz26" Jul 07 12:48:54.962436 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:54.962415 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-bzhg7" Jul 07 12:48:54.990075 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:54.990047 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:55.052049 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.052009 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz"] Jul 07 12:48:55.064032 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.064000 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq"] Jul 07 12:48:55.064249 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.064117 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" Jul 07 12:48:55.067179 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.067153 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 12:48:55.067179 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.067174 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-fpdcl\"" Jul 07 12:48:55.067370 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.067256 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" Jul 07 12:48:55.072973 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.072953 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 12:48:55.075560 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.075534 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8sc74\"" Jul 07 12:48:55.088107 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.087827 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq"] Jul 07 12:48:55.093165 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.093018 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz"] Jul 07 12:48:55.134735 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.134705 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-68mm9"] Jul 07 12:48:55.134919 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.134896 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" podUID="8524c624-9c9f-4aaf-9848-d57ae754453d" containerName="discovery" containerID="cri-o://48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423" gracePeriod=30 Jul 07 12:48:55.183324 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.183295 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:55.185441 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.185417 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51726836-56b5-40bf-a16e-766b86b9d750-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mr8fq\" (UID: \"51726836-56b5-40bf-a16e-766b86b9d750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" Jul 07 12:48:55.185562 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.185449 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqggn\" (UniqueName: \"kubernetes.io/projected/51726836-56b5-40bf-a16e-766b86b9d750-kube-api-access-hqggn\") pod \"llmisvc-controller-manager-777c6bd7d5-mr8fq\" (UID: \"51726836-56b5-40bf-a16e-766b86b9d750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" Jul 07 12:48:55.185562 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.185542 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc4hm\" (UniqueName: \"kubernetes.io/projected/d68a452b-a3eb-41a4-ad62-13792602001d-kube-api-access-zc4hm\") pod \"kserve-controller-manager-d6f4b65b8-xkgpz\" (UID: \"d68a452b-a3eb-41a4-ad62-13792602001d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" Jul 07 12:48:55.185677 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.185593 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d68a452b-a3eb-41a4-ad62-13792602001d-cert\") pod \"kserve-controller-manager-d6f4b65b8-xkgpz\" (UID: \"d68a452b-a3eb-41a4-ad62-13792602001d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" Jul 07 12:48:55.286923 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.286889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc4hm\" (UniqueName: \"kubernetes.io/projected/d68a452b-a3eb-41a4-ad62-13792602001d-kube-api-access-zc4hm\") pod \"kserve-controller-manager-d6f4b65b8-xkgpz\" (UID: \"d68a452b-a3eb-41a4-ad62-13792602001d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" Jul 07 12:48:55.287085 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.286954 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d68a452b-a3eb-41a4-ad62-13792602001d-cert\") pod \"kserve-controller-manager-d6f4b65b8-xkgpz\" (UID: \"d68a452b-a3eb-41a4-ad62-13792602001d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" Jul 07 12:48:55.287085 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.287073 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51726836-56b5-40bf-a16e-766b86b9d750-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mr8fq\" (UID: \"51726836-56b5-40bf-a16e-766b86b9d750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" Jul 07 12:48:55.287197 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.287100 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqggn\" (UniqueName: \"kubernetes.io/projected/51726836-56b5-40bf-a16e-766b86b9d750-kube-api-access-hqggn\") pod \"llmisvc-controller-manager-777c6bd7d5-mr8fq\" (UID: \"51726836-56b5-40bf-a16e-766b86b9d750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" Jul 07 12:48:55.289303 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.289280 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d68a452b-a3eb-41a4-ad62-13792602001d-cert\") pod \"kserve-controller-manager-d6f4b65b8-xkgpz\" (UID: \"d68a452b-a3eb-41a4-ad62-13792602001d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" Jul 07 12:48:55.289393 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.289343 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51726836-56b5-40bf-a16e-766b86b9d750-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mr8fq\" (UID: \"51726836-56b5-40bf-a16e-766b86b9d750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" Jul 07 12:48:55.291054 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.291029 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:55.300283 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.300263 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqggn\" (UniqueName: \"kubernetes.io/projected/51726836-56b5-40bf-a16e-766b86b9d750-kube-api-access-hqggn\") pod \"llmisvc-controller-manager-777c6bd7d5-mr8fq\" (UID: \"51726836-56b5-40bf-a16e-766b86b9d750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" Jul 07 12:48:55.304330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.304305 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc4hm\" (UniqueName: \"kubernetes.io/projected/d68a452b-a3eb-41a4-ad62-13792602001d-kube-api-access-zc4hm\") pod \"kserve-controller-manager-d6f4b65b8-xkgpz\" (UID: \"d68a452b-a3eb-41a4-ad62-13792602001d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" Jul 07 12:48:55.377144 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.377121 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:55.378961 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.378944 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" Jul 07 12:48:55.384667 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.384647 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" Jul 07 12:48:55.408124 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.408041 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:48:55.490243 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.490058 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-dns-cert\") pod \"8524c624-9c9f-4aaf-9848-d57ae754453d\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " Jul 07 12:48:55.490243 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.490108 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8524c624-9c9f-4aaf-9848-d57ae754453d-local-certs\") pod \"8524c624-9c9f-4aaf-9848-d57ae754453d\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " Jul 07 12:48:55.490243 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.490145 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-token\") pod \"8524c624-9c9f-4aaf-9848-d57ae754453d\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " Jul 07 12:48:55.490243 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.490180 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-kubeconfig\") pod \"8524c624-9c9f-4aaf-9848-d57ae754453d\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " Jul 07 12:48:55.490243 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.490239 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxwxj\" (UniqueName: \"kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-kube-api-access-sxwxj\") pod \"8524c624-9c9f-4aaf-9848-d57ae754453d\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " Jul 07 12:48:55.490595 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.490327 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-cacerts\") pod \"8524c624-9c9f-4aaf-9848-d57ae754453d\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " Jul 07 12:48:55.490595 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.490389 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-ca-configmap\") pod \"8524c624-9c9f-4aaf-9848-d57ae754453d\" (UID: \"8524c624-9c9f-4aaf-9848-d57ae754453d\") " Jul 07 12:48:55.491898 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.490979 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "8524c624-9c9f-4aaf-9848-d57ae754453d" (UID: "8524c624-9c9f-4aaf-9848-d57ae754453d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:48:55.493681 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.493627 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-token" (OuterVolumeSpecName: "istio-token") pod "8524c624-9c9f-4aaf-9848-d57ae754453d" (UID: "8524c624-9c9f-4aaf-9848-d57ae754453d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:48:55.494351 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.494263 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8524c624-9c9f-4aaf-9848-d57ae754453d-local-certs" (OuterVolumeSpecName: "local-certs") pod "8524c624-9c9f-4aaf-9848-d57ae754453d" (UID: "8524c624-9c9f-4aaf-9848-d57ae754453d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:48:55.494811 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.494742 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-cacerts" (OuterVolumeSpecName: "cacerts") pod "8524c624-9c9f-4aaf-9848-d57ae754453d" (UID: "8524c624-9c9f-4aaf-9848-d57ae754453d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:48:55.495809 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.495769 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "8524c624-9c9f-4aaf-9848-d57ae754453d" (UID: "8524c624-9c9f-4aaf-9848-d57ae754453d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:48:55.496393 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.496351 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "8524c624-9c9f-4aaf-9848-d57ae754453d" (UID: "8524c624-9c9f-4aaf-9848-d57ae754453d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:48:55.496648 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.496609 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-kube-api-access-sxwxj" (OuterVolumeSpecName: "kube-api-access-sxwxj") pod "8524c624-9c9f-4aaf-9848-d57ae754453d" (UID: "8524c624-9c9f-4aaf-9848-d57ae754453d"). InnerVolumeSpecName "kube-api-access-sxwxj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:48:55.566203 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:48:55.566160 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68a452b_a3eb_41a4_ad62_13792602001d.slice/crio-3e84d361f68b6e0de6d9cd834084ef7e23f693f0fafe6dbe9cd25e7369c3c840 WatchSource:0}: Error finding container 3e84d361f68b6e0de6d9cd834084ef7e23f693f0fafe6dbe9cd25e7369c3c840: Status 404 returned error can't find the container with id 3e84d361f68b6e0de6d9cd834084ef7e23f693f0fafe6dbe9cd25e7369c3c840 Jul 07 12:48:55.569061 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.569036 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz"] Jul 07 12:48:55.591289 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.591263 2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-ca-configmap\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:48:55.591289 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.591288 2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-csr-dns-cert\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:48:55.591289 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.591299 2574 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8524c624-9c9f-4aaf-9848-d57ae754453d-local-certs\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:48:55.591454 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.591308 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-token\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:48:55.591454 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.591316 2574 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-istio-kubeconfig\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:48:55.591454 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.591324 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxwxj\" (UniqueName: \"kubernetes.io/projected/8524c624-9c9f-4aaf-9848-d57ae754453d-kube-api-access-sxwxj\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:48:55.591454 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.591333 2574 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8524c624-9c9f-4aaf-9848-d57ae754453d-cacerts\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:48:55.594214 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:55.594194 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq"] Jul 07 12:48:55.595995 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:48:55.595966 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod51726836_56b5_40bf_a16e_766b86b9d750.slice/crio-71bf59e97169b852ece2bd1820353346965cc016446a8272d16ebd4933bc7473 WatchSource:0}: Error finding container 71bf59e97169b852ece2bd1820353346965cc016446a8272d16ebd4933bc7473: Status 404 returned error can't find the container with id 71bf59e97169b852ece2bd1820353346965cc016446a8272d16ebd4933bc7473 Jul 07 12:48:56.011761 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:56.011723 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" event={"ID":"d68a452b-a3eb-41a4-ad62-13792602001d","Type":"ContainerStarted","Data":"3e84d361f68b6e0de6d9cd834084ef7e23f693f0fafe6dbe9cd25e7369c3c840"} Jul 07 12:48:56.013147 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:56.013119 2574 generic.go:356] "Generic (PLEG): container finished" podID="8524c624-9c9f-4aaf-9848-d57ae754453d" containerID="48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423" exitCode=0 Jul 07 12:48:56.013274 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:56.013165 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" Jul 07 12:48:56.013274 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:56.013204 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" event={"ID":"8524c624-9c9f-4aaf-9848-d57ae754453d","Type":"ContainerDied","Data":"48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423"} Jul 07 12:48:56.013274 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:56.013251 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-68mm9" event={"ID":"8524c624-9c9f-4aaf-9848-d57ae754453d","Type":"ContainerDied","Data":"78273137a04a7173b78ea8e94c4256340a2091fce99133e83f0146a20e91fd63"} Jul 07 12:48:56.013400 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:56.013277 2574 scope.go:117] "RemoveContainer" containerID="48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423" Jul 07 12:48:56.014510 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:56.014479 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" event={"ID":"51726836-56b5-40bf-a16e-766b86b9d750","Type":"ContainerStarted","Data":"71bf59e97169b852ece2bd1820353346965cc016446a8272d16ebd4933bc7473"} Jul 07 12:48:56.022640 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:56.022471 2574 scope.go:117] "RemoveContainer" containerID="48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423" Jul 07 12:48:56.023025 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:48:56.022992 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423\": container with ID starting with 48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423 not found: ID does not exist" containerID="48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423" Jul 07 12:48:56.023142 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:56.023031 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423"} err="failed to get container status \"48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423\": rpc error: code = NotFound desc = could not find container \"48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423\": container with ID starting with 48c44c7e8850e387e532aa22330ee73b960345d67f32b5f932077f0883e85423 not found: ID does not exist" Jul 07 12:48:56.077289 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:56.077252 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-68mm9"] Jul 07 12:48:56.096473 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:56.096447 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-68mm9"] Jul 07 12:48:57.189320 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:57.189289 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8524c624-9c9f-4aaf-9848-d57ae754453d" path="/var/lib/kubelet/pods/8524c624-9c9f-4aaf-9848-d57ae754453d/volumes" Jul 07 12:48:59.032574 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:59.032537 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" event={"ID":"d68a452b-a3eb-41a4-ad62-13792602001d","Type":"ContainerStarted","Data":"aa8c50f349d077f68252e8f6db485bab421b63325c81229ffbf4c3c7c866e400"} Jul 07 12:48:59.033054 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:59.032682 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" Jul 07 12:48:59.066676 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:48:59.066629 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" podStartSLOduration=1.5529937679999999 podStartE2EDuration="4.066612707s" totalImagesPullingTime="2.513618939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:59.065318084 +0000 UTC m=+386.509361264" watchObservedRunningTime="2026-07-07 12:48:59.066612707 +0000 UTC m=+386.510655885" Jul 07 12:49:00.038528 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:00.038500 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" event={"ID":"51726836-56b5-40bf-a16e-766b86b9d750","Type":"ContainerStarted","Data":"596fe867d0a105ab55631865954eb9b7025dc437ecfc27be247f76d9ded36901"} Jul 07 12:49:00.038929 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:00.038657 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" Jul 07 12:49:00.176739 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:00.176683 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" podStartSLOduration=1.266316636 podStartE2EDuration="5.176667875s" totalImagesPullingTime="3.910351239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:00.176027793 +0000 UTC m=+387.620070993" watchObservedRunningTime="2026-07-07 12:49:00.176667875 +0000 UTC m=+387.620711053" Jul 07 12:49:30.043660 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:30.043631 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xkgpz" Jul 07 12:49:31.045627 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:31.045599 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mr8fq" Jul 07 12:49:38.662799 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.662757 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:49:38.725835 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.725803 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2pfn9"] Jul 07 12:49:38.726219 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.726196 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8524c624-9c9f-4aaf-9848-d57ae754453d" containerName="discovery" Jul 07 12:49:38.726219 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.726211 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8524c624-9c9f-4aaf-9848-d57ae754453d" containerName="discovery" Jul 07 12:49:38.726422 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.726271 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8524c624-9c9f-4aaf-9848-d57ae754453d" containerName="discovery" Jul 07 12:49:38.731014 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.730985 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" Jul 07 12:49:38.738266 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.738239 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 12:49:38.739899 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.739572 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 12:49:38.739978 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.739915 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-wvh6s\"" Jul 07 12:49:38.755615 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.755579 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2lln\" (UniqueName: \"kubernetes.io/projected/d5a19363-be76-4004-9c99-4e0463515f3b-kube-api-access-c2lln\") pod \"kuadrant-operator-catalog-2pfn9\" (UID: \"d5a19363-be76-4004-9c99-4e0463515f3b\") " pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" Jul 07 12:49:38.767628 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.767599 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2pfn9"] Jul 07 12:49:38.809648 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.809595 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:49:38.856750 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.856722 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2lln\" (UniqueName: \"kubernetes.io/projected/d5a19363-be76-4004-9c99-4e0463515f3b-kube-api-access-c2lln\") pod \"kuadrant-operator-catalog-2pfn9\" (UID: \"d5a19363-be76-4004-9c99-4e0463515f3b\") " pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" Jul 07 12:49:38.867669 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.867646 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2lln\" (UniqueName: \"kubernetes.io/projected/d5a19363-be76-4004-9c99-4e0463515f3b-kube-api-access-c2lln\") pod \"kuadrant-operator-catalog-2pfn9\" (UID: \"d5a19363-be76-4004-9c99-4e0463515f3b\") " pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" Jul 07 12:49:38.915072 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:38.914979 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:49:39.042100 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.042064 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2pfn9"] Jul 07 12:49:39.042329 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.042316 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" Jul 07 12:49:39.175944 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.175911 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2pfn9"] Jul 07 12:49:39.177434 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:49:39.177409 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a19363_be76_4004_9c99_4e0463515f3b.slice/crio-302286ac383c9dbaad98459b3c7d3374d9f2b3a370739d07f2ad7a890c01fe79 WatchSource:0}: Error finding container 302286ac383c9dbaad98459b3c7d3374d9f2b3a370739d07f2ad7a890c01fe79: Status 404 returned error can't find the container with id 302286ac383c9dbaad98459b3c7d3374d9f2b3a370739d07f2ad7a890c01fe79 Jul 07 12:49:39.234859 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.234826 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" event={"ID":"d5a19363-be76-4004-9c99-4e0463515f3b","Type":"ContainerStarted","Data":"302286ac383c9dbaad98459b3c7d3374d9f2b3a370739d07f2ad7a890c01fe79"} Jul 07 12:49:39.261816 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.261786 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qq7mv"] Jul 07 12:49:39.264897 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.264881 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" Jul 07 12:49:39.284065 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.284034 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qq7mv"] Jul 07 12:49:39.361601 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.361572 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzc7k\" (UniqueName: \"kubernetes.io/projected/4b2d4000-e605-4dda-9574-a7f5879e6530-kube-api-access-xzc7k\") pod \"kuadrant-operator-catalog-qq7mv\" (UID: \"4b2d4000-e605-4dda-9574-a7f5879e6530\") " pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" Jul 07 12:49:39.462850 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.462821 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzc7k\" (UniqueName: \"kubernetes.io/projected/4b2d4000-e605-4dda-9574-a7f5879e6530-kube-api-access-xzc7k\") pod \"kuadrant-operator-catalog-qq7mv\" (UID: \"4b2d4000-e605-4dda-9574-a7f5879e6530\") " pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" Jul 07 12:49:39.473299 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.473278 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzc7k\" (UniqueName: \"kubernetes.io/projected/4b2d4000-e605-4dda-9574-a7f5879e6530-kube-api-access-xzc7k\") pod \"kuadrant-operator-catalog-qq7mv\" (UID: \"4b2d4000-e605-4dda-9574-a7f5879e6530\") " pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" Jul 07 12:49:39.574098 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.574068 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" Jul 07 12:49:39.702192 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:39.702167 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qq7mv"] Jul 07 12:49:39.703909 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:49:39.703883 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2d4000_e605_4dda_9574_a7f5879e6530.slice/crio-88ee4c3ec3df01681cd3686f94615c0c519c37feea00d7d27dcd212f15656c26 WatchSource:0}: Error finding container 88ee4c3ec3df01681cd3686f94615c0c519c37feea00d7d27dcd212f15656c26: Status 404 returned error can't find the container with id 88ee4c3ec3df01681cd3686f94615c0c519c37feea00d7d27dcd212f15656c26 Jul 07 12:49:40.244186 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:40.244144 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" event={"ID":"4b2d4000-e605-4dda-9574-a7f5879e6530","Type":"ContainerStarted","Data":"88ee4c3ec3df01681cd3686f94615c0c519c37feea00d7d27dcd212f15656c26"} Jul 07 12:49:42.255627 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:42.255585 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" event={"ID":"d5a19363-be76-4004-9c99-4e0463515f3b","Type":"ContainerStarted","Data":"aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f"} Jul 07 12:49:42.256140 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:42.255602 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" podUID="d5a19363-be76-4004-9c99-4e0463515f3b" containerName="registry-server" containerID="cri-o://aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f" gracePeriod=2 Jul 07 12:49:42.257135 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:42.257112 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" event={"ID":"4b2d4000-e605-4dda-9574-a7f5879e6530","Type":"ContainerStarted","Data":"f3448b82c34a49800e195b34b5edeec86551ce6d540a7ae89800872098c29b13"} Jul 07 12:49:42.299434 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:42.299382 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" podStartSLOduration=1.7844812380000001 podStartE2EDuration="3.29936399s" totalImagesPullingTime="1.514882752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:42.299156859 +0000 UTC m=+429.743200058" watchObservedRunningTime="2026-07-07 12:49:42.29936399 +0000 UTC m=+429.743407170" Jul 07 12:49:42.300299 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:42.300264 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" podStartSLOduration=2.255253249 podStartE2EDuration="4.300253492s" totalImagesPullingTime="2.045000243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:49:42.281891472 +0000 UTC m=+429.725934669" watchObservedRunningTime="2026-07-07 12:49:42.300253492 +0000 UTC m=+429.744296671" Jul 07 12:49:42.492953 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:42.492931 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" Jul 07 12:49:42.591674 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:42.591590 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2lln\" (UniqueName: \"kubernetes.io/projected/d5a19363-be76-4004-9c99-4e0463515f3b-kube-api-access-c2lln\") pod \"d5a19363-be76-4004-9c99-4e0463515f3b\" (UID: \"d5a19363-be76-4004-9c99-4e0463515f3b\") " Jul 07 12:49:42.593759 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:42.593729 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5a19363-be76-4004-9c99-4e0463515f3b-kube-api-access-c2lln" (OuterVolumeSpecName: "kube-api-access-c2lln") pod "d5a19363-be76-4004-9c99-4e0463515f3b" (UID: "d5a19363-be76-4004-9c99-4e0463515f3b"). InnerVolumeSpecName "kube-api-access-c2lln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:49:42.693420 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:42.693385 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2lln\" (UniqueName: \"kubernetes.io/projected/d5a19363-be76-4004-9c99-4e0463515f3b-kube-api-access-c2lln\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:49:43.263454 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:43.263417 2574 generic.go:356] "Generic (PLEG): container finished" podID="d5a19363-be76-4004-9c99-4e0463515f3b" containerID="aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f" exitCode=0 Jul 07 12:49:43.263859 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:43.263468 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" Jul 07 12:49:43.263859 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:43.263502 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" event={"ID":"d5a19363-be76-4004-9c99-4e0463515f3b","Type":"ContainerDied","Data":"aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f"} Jul 07 12:49:43.263859 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:43.263557 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2pfn9" event={"ID":"d5a19363-be76-4004-9c99-4e0463515f3b","Type":"ContainerDied","Data":"302286ac383c9dbaad98459b3c7d3374d9f2b3a370739d07f2ad7a890c01fe79"} Jul 07 12:49:43.263859 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:43.263584 2574 scope.go:117] "RemoveContainer" containerID="aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f" Jul 07 12:49:43.273027 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:43.273012 2574 scope.go:117] "RemoveContainer" containerID="aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f" Jul 07 12:49:43.273280 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:49:43.273257 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f\": container with ID starting with aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f not found: ID does not exist" containerID="aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f" Jul 07 12:49:43.273378 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:43.273283 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f"} err="failed to get container status \"aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f\": rpc error: code = NotFound desc = could not find container \"aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f\": container with ID starting with aecf7229a3c15626dbe100ccbc8599ff1b53ed67fa323b70f09b5de3a244cf3f not found: ID does not exist" Jul 07 12:49:43.286152 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:43.286126 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2pfn9"] Jul 07 12:49:43.288658 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:43.288638 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2pfn9"] Jul 07 12:49:45.189081 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:45.189050 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5a19363-be76-4004-9c99-4e0463515f3b" path="/var/lib/kubelet/pods/d5a19363-be76-4004-9c99-4e0463515f3b/volumes" Jul 07 12:49:49.574218 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:49.574189 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" Jul 07 12:49:49.574596 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:49.574238 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" Jul 07 12:49:49.596559 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:49.596532 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" Jul 07 12:49:50.321910 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:49:50.321879 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-qq7mv" Jul 07 12:50:02.475398 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:02.475303 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:07.566172 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:07.566120 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:12.149446 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.149384 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q"] Jul 07 12:50:12.152520 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.150449 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5a19363-be76-4004-9c99-4e0463515f3b" containerName="registry-server" Jul 07 12:50:12.152520 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.150484 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5a19363-be76-4004-9c99-4e0463515f3b" containerName="registry-server" Jul 07 12:50:12.152520 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.150759 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5a19363-be76-4004-9c99-4e0463515f3b" containerName="registry-server" Jul 07 12:50:12.156507 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.153865 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:12.158242 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.158220 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wht5w\"" Jul 07 12:50:12.168210 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.168189 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q"] Jul 07 12:50:12.256199 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.256172 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" (UID: \"5d7828c1-fb1f-44ef-9673-f6fda2ec4588\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:12.256338 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.256234 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj8js\" (UniqueName: \"kubernetes.io/projected/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-kube-api-access-nj8js\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" (UID: \"5d7828c1-fb1f-44ef-9673-f6fda2ec4588\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:12.357379 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.357350 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj8js\" (UniqueName: \"kubernetes.io/projected/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-kube-api-access-nj8js\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" (UID: \"5d7828c1-fb1f-44ef-9673-f6fda2ec4588\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:12.357491 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.357426 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" (UID: \"5d7828c1-fb1f-44ef-9673-f6fda2ec4588\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:12.357794 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.357777 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" (UID: \"5d7828c1-fb1f-44ef-9673-f6fda2ec4588\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:12.370059 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.370029 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj8js\" (UniqueName: \"kubernetes.io/projected/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-kube-api-access-nj8js\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" (UID: \"5d7828c1-fb1f-44ef-9673-f6fda2ec4588\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:12.464760 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.464738 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:12.598114 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.598092 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q"] Jul 07 12:50:12.600032 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:50:12.600001 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7828c1_fb1f_44ef_9673_f6fda2ec4588.slice/crio-9399b7be343c2999d2ba832edba1f72f7dc61f7e5a78cda8775981c1a09cf293 WatchSource:0}: Error finding container 9399b7be343c2999d2ba832edba1f72f7dc61f7e5a78cda8775981c1a09cf293: Status 404 returned error can't find the container with id 9399b7be343c2999d2ba832edba1f72f7dc61f7e5a78cda8775981c1a09cf293 Jul 07 12:50:12.840623 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.840553 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dc9556d4b-8k5rm"] Jul 07 12:50:12.843577 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.843557 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.846892 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.846846 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 12:50:12.846892 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.846868 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 12:50:12.846892 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.846890 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 12:50:12.847166 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.846853 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8vd9j\"" Jul 07 12:50:12.847166 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.847008 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 12:50:12.847166 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.847119 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 12:50:12.847330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.847254 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 12:50:12.847330 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.847313 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 12:50:12.852967 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.852947 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 12:50:12.860420 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.860400 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-trusted-ca-bundle\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.860504 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.860439 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-oauth-serving-cert\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.860504 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.860455 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxlch\" (UniqueName: \"kubernetes.io/projected/440b7cb4-d1d7-416a-9a42-fe6960cb679b-kube-api-access-jxlch\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.860504 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.860473 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/440b7cb4-d1d7-416a-9a42-fe6960cb679b-console-serving-cert\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.860673 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.860566 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/440b7cb4-d1d7-416a-9a42-fe6960cb679b-console-oauth-config\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.860673 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.860629 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-service-ca\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.860784 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.860680 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-console-config\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.869302 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.869281 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dc9556d4b-8k5rm"] Jul 07 12:50:12.941501 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.941471 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:12.961867 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.961844 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-console-config\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.961971 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.961878 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-trusted-ca-bundle\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.961971 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.961914 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-oauth-serving-cert\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.961971 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.961939 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxlch\" (UniqueName: \"kubernetes.io/projected/440b7cb4-d1d7-416a-9a42-fe6960cb679b-kube-api-access-jxlch\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.962250 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.962231 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/440b7cb4-d1d7-416a-9a42-fe6960cb679b-console-serving-cert\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.962318 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.962288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/440b7cb4-d1d7-416a-9a42-fe6960cb679b-console-oauth-config\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.962375 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.962364 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-service-ca\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.962564 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.962543 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-console-config\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.962678 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.962658 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-oauth-serving-cert\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.962782 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.962770 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-trusted-ca-bundle\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.962924 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.962907 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/440b7cb4-d1d7-416a-9a42-fe6960cb679b-service-ca\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.964736 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.964717 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/440b7cb4-d1d7-416a-9a42-fe6960cb679b-console-serving-cert\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.964823 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.964735 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/440b7cb4-d1d7-416a-9a42-fe6960cb679b-console-oauth-config\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:12.976120 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:12.976101 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxlch\" (UniqueName: \"kubernetes.io/projected/440b7cb4-d1d7-416a-9a42-fe6960cb679b-kube-api-access-jxlch\") pod \"console-dc9556d4b-8k5rm\" (UID: \"440b7cb4-d1d7-416a-9a42-fe6960cb679b\") " pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:13.035007 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:13.034978 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:13.153617 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:13.153527 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:13.192552 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:13.192517 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:13.300059 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:13.300035 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dc9556d4b-8k5rm"] Jul 07 12:50:13.301594 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:50:13.301567 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440b7cb4_d1d7_416a_9a42_fe6960cb679b.slice/crio-c73ebe7e54fa4ae3b3e8ac64567a3c9b7f3851aefb48f8b92c4be2b68ec51032 WatchSource:0}: Error finding container c73ebe7e54fa4ae3b3e8ac64567a3c9b7f3851aefb48f8b92c4be2b68ec51032: Status 404 returned error can't find the container with id c73ebe7e54fa4ae3b3e8ac64567a3c9b7f3851aefb48f8b92c4be2b68ec51032 Jul 07 12:50:13.412448 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:13.412345 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dc9556d4b-8k5rm" event={"ID":"440b7cb4-d1d7-416a-9a42-fe6960cb679b","Type":"ContainerStarted","Data":"985fde968d8ce938e11e6b2c37f2d25f773b387e0a5ed96b27c3e5240ef0d965"} Jul 07 12:50:13.412448 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:13.412410 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dc9556d4b-8k5rm" event={"ID":"440b7cb4-d1d7-416a-9a42-fe6960cb679b","Type":"ContainerStarted","Data":"c73ebe7e54fa4ae3b3e8ac64567a3c9b7f3851aefb48f8b92c4be2b68ec51032"} Jul 07 12:50:13.414073 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:13.414050 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" event={"ID":"5d7828c1-fb1f-44ef-9673-f6fda2ec4588","Type":"ContainerStarted","Data":"9399b7be343c2999d2ba832edba1f72f7dc61f7e5a78cda8775981c1a09cf293"} Jul 07 12:50:13.446865 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:13.446813 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dc9556d4b-8k5rm" podStartSLOduration=1.446798553 podStartE2EDuration="1.446798553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:50:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:50:13.444917314 +0000 UTC m=+460.888960495" watchObservedRunningTime="2026-07-07 12:50:13.446798553 +0000 UTC m=+460.890841732" Jul 07 12:50:14.026818 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:14.026788 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:15.078108 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.077930 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr"] Jul 07 12:50:15.085462 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.085407 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr" Jul 07 12:50:15.094336 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.089940 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 07 12:50:15.094336 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.090904 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-97t6d\"" Jul 07 12:50:15.098945 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.098926 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr"] Jul 07 12:50:15.185683 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.185652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gj2c\" (UniqueName: \"kubernetes.io/projected/005a2d2b-5a6e-42d5-9e0d-f7c76a89c436-kube-api-access-9gj2c\") pod \"dns-operator-controller-manager-864dd78c4c-nd6gr\" (UID: \"005a2d2b-5a6e-42d5-9e0d-f7c76a89c436\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr" Jul 07 12:50:15.267314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.267283 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:15.286621 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.286584 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gj2c\" (UniqueName: \"kubernetes.io/projected/005a2d2b-5a6e-42d5-9e0d-f7c76a89c436-kube-api-access-9gj2c\") pod \"dns-operator-controller-manager-864dd78c4c-nd6gr\" (UID: \"005a2d2b-5a6e-42d5-9e0d-f7c76a89c436\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr" Jul 07 12:50:15.300196 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.300144 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gj2c\" (UniqueName: \"kubernetes.io/projected/005a2d2b-5a6e-42d5-9e0d-f7c76a89c436-kube-api-access-9gj2c\") pod \"dns-operator-controller-manager-864dd78c4c-nd6gr\" (UID: \"005a2d2b-5a6e-42d5-9e0d-f7c76a89c436\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr" Jul 07 12:50:15.413809 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.413724 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr" Jul 07 12:50:15.561397 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.561369 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr"] Jul 07 12:50:15.562220 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:50:15.562180 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005a2d2b_5a6e_42d5_9e0d_f7c76a89c436.slice/crio-696ef522faba8c1709258ba0665c5363bc1c8dfac9dd4a3690ee44faaabf4e4a WatchSource:0}: Error finding container 696ef522faba8c1709258ba0665c5363bc1c8dfac9dd4a3690ee44faaabf4e4a: Status 404 returned error can't find the container with id 696ef522faba8c1709258ba0665c5363bc1c8dfac9dd4a3690ee44faaabf4e4a Jul 07 12:50:15.596130 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:15.596100 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:16.188541 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:16.188506 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:16.432579 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:16.432543 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr" event={"ID":"005a2d2b-5a6e-42d5-9e0d-f7c76a89c436","Type":"ContainerStarted","Data":"696ef522faba8c1709258ba0665c5363bc1c8dfac9dd4a3690ee44faaabf4e4a"} Jul 07 12:50:18.144030 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:18.143995 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj"] Jul 07 12:50:18.147748 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:18.147725 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj" Jul 07 12:50:18.151159 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:18.151139 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-2dqd5\"" Jul 07 12:50:18.182101 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:18.182067 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj"] Jul 07 12:50:18.216562 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:18.216531 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnvhz\" (UniqueName: \"kubernetes.io/projected/ae483d2d-ad2b-4afd-bc5e-e4c7726a4819-kube-api-access-nnvhz\") pod \"authorino-operator-75cc8d5cb7-bwhpj\" (UID: \"ae483d2d-ad2b-4afd-bc5e-e4c7726a4819\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj" Jul 07 12:50:18.260832 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:18.260803 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:18.318167 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:18.318121 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnvhz\" (UniqueName: \"kubernetes.io/projected/ae483d2d-ad2b-4afd-bc5e-e4c7726a4819-kube-api-access-nnvhz\") pod \"authorino-operator-75cc8d5cb7-bwhpj\" (UID: \"ae483d2d-ad2b-4afd-bc5e-e4c7726a4819\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj" Jul 07 12:50:18.341451 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:18.341412 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnvhz\" (UniqueName: \"kubernetes.io/projected/ae483d2d-ad2b-4afd-bc5e-e4c7726a4819-kube-api-access-nnvhz\") pod \"authorino-operator-75cc8d5cb7-bwhpj\" (UID: \"ae483d2d-ad2b-4afd-bc5e-e4c7726a4819\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj" Jul 07 12:50:18.395871 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:18.395796 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:18.460343 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:18.460303 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj" Jul 07 12:50:18.678031 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:18.678003 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj"] Jul 07 12:50:18.679836 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:50:18.679814 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae483d2d_ad2b_4afd_bc5e_e4c7726a4819.slice/crio-814bab64d368c1b50a96a510390230280db8a036876a10f17fbc5f11fbdee9f6 WatchSource:0}: Error finding container 814bab64d368c1b50a96a510390230280db8a036876a10f17fbc5f11fbdee9f6: Status 404 returned error can't find the container with id 814bab64d368c1b50a96a510390230280db8a036876a10f17fbc5f11fbdee9f6 Jul 07 12:50:19.211051 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:19.211018 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:19.454408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:19.454277 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr" event={"ID":"005a2d2b-5a6e-42d5-9e0d-f7c76a89c436","Type":"ContainerStarted","Data":"f2c8a2d553a5719952c77219224b35839c7a8086cf5a288eec8a2e797e85cbe3"} Jul 07 12:50:19.454408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:19.454365 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr" Jul 07 12:50:19.457052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:19.456655 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" event={"ID":"5d7828c1-fb1f-44ef-9673-f6fda2ec4588","Type":"ContainerStarted","Data":"c4b6efbb1a9eec48c5d4524e63bb0e7d74816f813d8d43d7469dad4871944903"} Jul 07 12:50:19.457052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:19.456759 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:19.458338 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:19.458311 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj" event={"ID":"ae483d2d-ad2b-4afd-bc5e-e4c7726a4819","Type":"ContainerStarted","Data":"814bab64d368c1b50a96a510390230280db8a036876a10f17fbc5f11fbdee9f6"} Jul 07 12:50:19.474643 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:19.474293 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr" podStartSLOduration=1.460570224 podStartE2EDuration="4.474276597s" totalImagesPullingTime="3.013706373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:50:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:50:19.473367618 +0000 UTC m=+466.917410823" watchObservedRunningTime="2026-07-07 12:50:19.474276597 +0000 UTC m=+466.918319776" Jul 07 12:50:19.499115 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:19.499058 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" podStartSLOduration=1.5602167200000001 podStartE2EDuration="7.499046715s" totalImagesPullingTime="5.938829995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:50:19.498866244 +0000 UTC m=+466.942909427" watchObservedRunningTime="2026-07-07 12:50:19.499046715 +0000 UTC m=+466.943089986" Jul 07 12:50:21.014038 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.014002 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:21.058019 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.057989 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s"] Jul 07 12:50:21.061332 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.061312 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:21.065228 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.065180 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6dg5p\"" Jul 07 12:50:21.075823 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.075798 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s"] Jul 07 12:50:21.146558 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.146449 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwt8z\" (UniqueName: \"kubernetes.io/projected/5bfa2baa-f815-4056-84ac-53c5cd5f47c0-kube-api-access-vwt8z\") pod \"limitador-operator-controller-manager-859d7b78bd-cts9s\" (UID: \"5bfa2baa-f815-4056-84ac-53c5cd5f47c0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:21.247953 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.247922 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwt8z\" (UniqueName: \"kubernetes.io/projected/5bfa2baa-f815-4056-84ac-53c5cd5f47c0-kube-api-access-vwt8z\") pod \"limitador-operator-controller-manager-859d7b78bd-cts9s\" (UID: \"5bfa2baa-f815-4056-84ac-53c5cd5f47c0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:21.260365 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.260335 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwt8z\" (UniqueName: \"kubernetes.io/projected/5bfa2baa-f815-4056-84ac-53c5cd5f47c0-kube-api-access-vwt8z\") pod \"limitador-operator-controller-manager-859d7b78bd-cts9s\" (UID: \"5bfa2baa-f815-4056-84ac-53c5cd5f47c0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:21.372100 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.372065 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:21.472581 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.472531 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj" event={"ID":"ae483d2d-ad2b-4afd-bc5e-e4c7726a4819","Type":"ContainerStarted","Data":"74d392b87d03e91d24afc5dfd5a31041a0bdeccc8c6e6a0fbf5d737c08b4344d"} Jul 07 12:50:21.472850 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.472826 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj" Jul 07 12:50:21.498271 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.498224 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj" podStartSLOduration=1.317763654 podStartE2EDuration="3.498208539s" totalImagesPullingTime="2.180444885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:50:21.496879011 +0000 UTC m=+468.940922189" watchObservedRunningTime="2026-07-07 12:50:21.498208539 +0000 UTC m=+468.942251717" Jul 07 12:50:21.512970 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.512950 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s"] Jul 07 12:50:21.514490 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:50:21.514457 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfa2baa_f815_4056_84ac_53c5cd5f47c0.slice/crio-9bd0ea0e5af8926ac94e578b25c459fda09a2df8ea80d21203f75c0221f4b88b WatchSource:0}: Error finding container 9bd0ea0e5af8926ac94e578b25c459fda09a2df8ea80d21203f75c0221f4b88b: Status 404 returned error can't find the container with id 9bd0ea0e5af8926ac94e578b25c459fda09a2df8ea80d21203f75c0221f4b88b Jul 07 12:50:21.959749 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:21.959712 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:22.053483 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:22.053439 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:22.479745 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:22.479706 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" event={"ID":"5bfa2baa-f815-4056-84ac-53c5cd5f47c0","Type":"ContainerStarted","Data":"9bd0ea0e5af8926ac94e578b25c459fda09a2df8ea80d21203f75c0221f4b88b"} Jul 07 12:50:23.031156 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:23.031121 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:23.129554 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:23.129511 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:23.154491 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:23.154464 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:23.154654 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:23.154527 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:23.159788 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:23.159765 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:23.486397 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:23.486367 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" event={"ID":"5bfa2baa-f815-4056-84ac-53c5cd5f47c0","Type":"ContainerStarted","Data":"43461d8367e8de7ab05667b2872802c93ca0b65f79355d4c0fc1551d89cdc3db"} Jul 07 12:50:23.486584 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:23.486560 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:23.490077 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:23.490056 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dc9556d4b-8k5rm" Jul 07 12:50:23.508591 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:23.508543 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" podStartSLOduration=0.759548539 podStartE2EDuration="2.508522221s" totalImagesPullingTime="1.748973682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:50:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:50:23.508256652 +0000 UTC m=+470.952299857" watchObservedRunningTime="2026-07-07 12:50:23.508522221 +0000 UTC m=+470.952565400" Jul 07 12:50:23.723297 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:23.723266 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:23.827747 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:23.827718 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:29.813860 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:29.813829 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:30.466202 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:30.466172 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:30.466361 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:30.466246 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nd6gr" Jul 07 12:50:30.615506 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:30.615476 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:30.716292 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:30.716217 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:31.606851 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:31.606820 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:31.785410 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:31.785383 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:32.198360 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.198327 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7"] Jul 07 12:50:32.201832 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.201815 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.216195 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.216174 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7"] Jul 07 12:50:32.293973 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.293941 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q"] Jul 07 12:50:32.294146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.294125 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" containerName="manager" containerID="cri-o://c4b6efbb1a9eec48c5d4524e63bb0e7d74816f813d8d43d7469dad4871944903" gracePeriod=2 Jul 07 12:50:32.299464 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.299438 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q"] Jul 07 12:50:32.299626 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.299598 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:32.313035 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.312997 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7"] Jul 07 12:50:32.313316 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.313295 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-dvlrl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" podUID="84194c6c-2f33-4f7e-9439-1ab37bc29b85" Jul 07 12:50:32.319938 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.319913 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz"] Jul 07 12:50:32.320534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.320512 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" containerName="manager" Jul 07 12:50:32.320621 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.320537 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" containerName="manager" Jul 07 12:50:32.320668 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.320646 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" containerName="manager" Jul 07 12:50:32.323890 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.323867 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:32.326150 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.326125 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7"] Jul 07 12:50:32.326261 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.326233 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:32.342181 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.342150 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvlrl\" (UniqueName: \"kubernetes.io/projected/84194c6c-2f33-4f7e-9439-1ab37bc29b85-kube-api-access-dvlrl\") pod \"kuadrant-operator-controller-manager-86878c769f-ktrv7\" (UID: \"84194c6c-2f33-4f7e-9439-1ab37bc29b85\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.342434 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.342230 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84194c6c-2f33-4f7e-9439-1ab37bc29b85-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ktrv7\" (UID: \"84194c6c-2f33-4f7e-9439-1ab37bc29b85\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.343470 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.343450 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz"] Jul 07 12:50:32.347789 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.347770 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp"] Jul 07 12:50:32.351091 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.351073 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 12:50:32.356296 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.356276 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s"] Jul 07 12:50:32.356461 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.356442 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" containerName="manager" containerID="cri-o://43461d8367e8de7ab05667b2872802c93ca0b65f79355d4c0fc1551d89cdc3db" gracePeriod=2 Jul 07 12:50:32.358494 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.358474 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:32.365723 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.365678 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s"] Jul 07 12:50:32.373195 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.373174 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp"] Jul 07 12:50:32.381373 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.381353 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh"] Jul 07 12:50:32.381880 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.381861 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" containerName="manager" Jul 07 12:50:32.381983 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.381966 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" containerName="manager" Jul 07 12:50:32.382127 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.382106 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" containerName="manager" Jul 07 12:50:32.385575 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.385547 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh" Jul 07 12:50:32.403668 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.403647 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh"] Jul 07 12:50:32.404130 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.404108 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cts9s\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:32.406207 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.406181 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:32.431544 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.431513 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cts9s\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:32.434101 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.434045 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:32.443022 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.443001 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6xrv\" (UniqueName: \"kubernetes.io/projected/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-kube-api-access-g6xrv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8l6gz\" (UID: \"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:32.443109 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.443042 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fjp8\" (UniqueName: \"kubernetes.io/projected/9b049194-41bb-400f-ba08-9d205c54ea82-kube-api-access-2fjp8\") pod \"limitador-operator-controller-manager-859d7b78bd-7dqxh\" (UID: \"9b049194-41bb-400f-ba08-9d205c54ea82\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh" Jul 07 12:50:32.443109 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.443086 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvlrl\" (UniqueName: \"kubernetes.io/projected/84194c6c-2f33-4f7e-9439-1ab37bc29b85-kube-api-access-dvlrl\") pod \"kuadrant-operator-controller-manager-86878c769f-ktrv7\" (UID: \"84194c6c-2f33-4f7e-9439-1ab37bc29b85\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.443226 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.443144 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2jznp\" (UID: \"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 12:50:32.443226 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.443178 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqhjj\" (UniqueName: \"kubernetes.io/projected/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-kube-api-access-gqhjj\") pod \"kuadrant-operator-controller-manager-86878c769f-2jznp\" (UID: \"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 12:50:32.443332 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.443234 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84194c6c-2f33-4f7e-9439-1ab37bc29b85-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ktrv7\" (UID: \"84194c6c-2f33-4f7e-9439-1ab37bc29b85\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.443332 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.443261 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8l6gz\" (UID: \"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:32.443541 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.443525 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84194c6c-2f33-4f7e-9439-1ab37bc29b85-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ktrv7\" (UID: \"84194c6c-2f33-4f7e-9439-1ab37bc29b85\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.450717 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.450686 2574 projected.go:196] Error preparing data for projected volume kube-api-access-dvlrl for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-ktrv7" not found Jul 07 12:50:32.450806 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.450760 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84194c6c-2f33-4f7e-9439-1ab37bc29b85-kube-api-access-dvlrl podName:84194c6c-2f33-4f7e-9439-1ab37bc29b85 nodeName:}" failed. No retries permitted until 2026-07-07 12:50:32.950745675 +0000 UTC m=+480.394788832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dvlrl" (UniqueName: "kubernetes.io/projected/84194c6c-2f33-4f7e-9439-1ab37bc29b85-kube-api-access-dvlrl") pod "kuadrant-operator-controller-manager-86878c769f-ktrv7" (UID: "84194c6c-2f33-4f7e-9439-1ab37bc29b85") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-ktrv7" not found Jul 07 12:50:32.482570 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.482545 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwhpj" Jul 07 12:50:32.485060 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.485029 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cts9s\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:32.487568 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.487547 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:32.536396 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.536339 2574 generic.go:356] "Generic (PLEG): container finished" podID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" containerID="c4b6efbb1a9eec48c5d4524e63bb0e7d74816f813d8d43d7469dad4871944903" exitCode=0 Jul 07 12:50:32.539594 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.539267 2574 generic.go:356] "Generic (PLEG): container finished" podID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" containerID="43461d8367e8de7ab05667b2872802c93ca0b65f79355d4c0fc1551d89cdc3db" exitCode=0 Jul 07 12:50:32.539594 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.539332 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.542461 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.542273 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cts9s\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:32.545033 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.544494 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6xrv\" (UniqueName: \"kubernetes.io/projected/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-kube-api-access-g6xrv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8l6gz\" (UID: \"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:32.545033 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.544544 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fjp8\" (UniqueName: \"kubernetes.io/projected/9b049194-41bb-400f-ba08-9d205c54ea82-kube-api-access-2fjp8\") pod \"limitador-operator-controller-manager-859d7b78bd-7dqxh\" (UID: \"9b049194-41bb-400f-ba08-9d205c54ea82\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh" Jul 07 12:50:32.545033 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.544626 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2jznp\" (UID: \"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 12:50:32.545033 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.544854 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqhjj\" (UniqueName: \"kubernetes.io/projected/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-kube-api-access-gqhjj\") pod \"kuadrant-operator-controller-manager-86878c769f-2jznp\" (UID: \"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 12:50:32.545033 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.544906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8l6gz\" (UID: \"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:32.545365 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.545065 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2jznp\" (UID: \"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 12:50:32.545365 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.545209 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8l6gz\" (UID: \"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:32.545612 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.545584 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-ktrv7\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="84194c6c-2f33-4f7e-9439-1ab37bc29b85" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.547671 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.547651 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:32.548128 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.548111 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.551776 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.551738 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cts9s\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:32.554868 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.554775 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-ktrv7\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="84194c6c-2f33-4f7e-9439-1ab37bc29b85" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.555869 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.555847 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6xrv\" (UniqueName: \"kubernetes.io/projected/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-kube-api-access-g6xrv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8l6gz\" (UID: \"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:32.556980 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.556958 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:32.562011 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.561968 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fjp8\" (UniqueName: \"kubernetes.io/projected/9b049194-41bb-400f-ba08-9d205c54ea82-kube-api-access-2fjp8\") pod \"limitador-operator-controller-manager-859d7b78bd-7dqxh\" (UID: \"9b049194-41bb-400f-ba08-9d205c54ea82\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh" Jul 07 12:50:32.563367 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.563322 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqhjj\" (UniqueName: \"kubernetes.io/projected/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-kube-api-access-gqhjj\") pod \"kuadrant-operator-controller-manager-86878c769f-2jznp\" (UID: \"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 12:50:32.608447 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.608424 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:32.610954 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.610924 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cts9s\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:32.611268 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.611253 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:32.613220 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.613197 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-ktrv7\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="84194c6c-2f33-4f7e-9439-1ab37bc29b85" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.615477 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.615458 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:32.617490 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.617473 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cts9s\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:32.619456 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.619438 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-ktrv7\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="84194c6c-2f33-4f7e-9439-1ab37bc29b85" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:32.621374 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:32.621358 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:32.645697 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.645675 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84194c6c-2f33-4f7e-9439-1ab37bc29b85-extensions-socket-volume\") pod \"84194c6c-2f33-4f7e-9439-1ab37bc29b85\" (UID: \"84194c6c-2f33-4f7e-9439-1ab37bc29b85\") " Jul 07 12:50:32.645937 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.645910 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84194c6c-2f33-4f7e-9439-1ab37bc29b85-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "84194c6c-2f33-4f7e-9439-1ab37bc29b85" (UID: "84194c6c-2f33-4f7e-9439-1ab37bc29b85"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:50:32.646064 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.646051 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvlrl\" (UniqueName: \"kubernetes.io/projected/84194c6c-2f33-4f7e-9439-1ab37bc29b85-kube-api-access-dvlrl\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:50:32.646106 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.646069 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84194c6c-2f33-4f7e-9439-1ab37bc29b85-extensions-socket-volume\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:50:32.716668 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.716642 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:32.724525 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.724506 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 12:50:32.731380 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.731361 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh" Jul 07 12:50:32.746541 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.746513 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-extensions-socket-volume\") pod \"5d7828c1-fb1f-44ef-9673-f6fda2ec4588\" (UID: \"5d7828c1-fb1f-44ef-9673-f6fda2ec4588\") " Jul 07 12:50:32.746661 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.746603 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwt8z\" (UniqueName: \"kubernetes.io/projected/5bfa2baa-f815-4056-84ac-53c5cd5f47c0-kube-api-access-vwt8z\") pod \"5bfa2baa-f815-4056-84ac-53c5cd5f47c0\" (UID: \"5bfa2baa-f815-4056-84ac-53c5cd5f47c0\") " Jul 07 12:50:32.746661 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.746640 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj8js\" (UniqueName: \"kubernetes.io/projected/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-kube-api-access-nj8js\") pod \"5d7828c1-fb1f-44ef-9673-f6fda2ec4588\" (UID: \"5d7828c1-fb1f-44ef-9673-f6fda2ec4588\") " Jul 07 12:50:32.747214 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.747187 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5d7828c1-fb1f-44ef-9673-f6fda2ec4588" (UID: "5d7828c1-fb1f-44ef-9673-f6fda2ec4588"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:50:32.749022 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.748994 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bfa2baa-f815-4056-84ac-53c5cd5f47c0-kube-api-access-vwt8z" (OuterVolumeSpecName: "kube-api-access-vwt8z") pod "5bfa2baa-f815-4056-84ac-53c5cd5f47c0" (UID: "5bfa2baa-f815-4056-84ac-53c5cd5f47c0"). InnerVolumeSpecName "kube-api-access-vwt8z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:50:32.749545 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.749520 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-kube-api-access-nj8js" (OuterVolumeSpecName: "kube-api-access-nj8js") pod "5d7828c1-fb1f-44ef-9673-f6fda2ec4588" (UID: "5d7828c1-fb1f-44ef-9673-f6fda2ec4588"). InnerVolumeSpecName "kube-api-access-nj8js". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:50:32.848517 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.848471 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-extensions-socket-volume\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:50:32.848517 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.848496 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwt8z\" (UniqueName: \"kubernetes.io/projected/5bfa2baa-f815-4056-84ac-53c5cd5f47c0-kube-api-access-vwt8z\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:50:32.848517 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.848507 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj8js\" (UniqueName: \"kubernetes.io/projected/5d7828c1-fb1f-44ef-9673-f6fda2ec4588-kube-api-access-nj8js\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:50:32.891166 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.891125 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz"] Jul 07 12:50:32.893796 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:50:32.893762 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ef5723_7308_4d9a_9d1c_57fa4aed14b6.slice/crio-713ce3add73e96d927c785db7fd04af1838d588673fe1165a8926b3520efed84 WatchSource:0}: Error finding container 713ce3add73e96d927c785db7fd04af1838d588673fe1165a8926b3520efed84: Status 404 returned error can't find the container with id 713ce3add73e96d927c785db7fd04af1838d588673fe1165a8926b3520efed84 Jul 07 12:50:32.927428 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.927402 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh"] Jul 07 12:50:32.937852 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:50:32.937825 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b049194_41bb_400f_ba08_9d205c54ea82.slice/crio-397389505d8f5dd534637b0ba9558e5f2457ef4516649dbf14e556d997e131b6 WatchSource:0}: Error finding container 397389505d8f5dd534637b0ba9558e5f2457ef4516649dbf14e556d997e131b6: Status 404 returned error can't find the container with id 397389505d8f5dd534637b0ba9558e5f2457ef4516649dbf14e556d997e131b6 Jul 07 12:50:32.948442 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:32.948417 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp"] Jul 07 12:50:32.956774 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:50:32.956753 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4ab2d6_c2ba_4e24_8240_6bb69d398a2f.slice/crio-27a8882b8f73395b7c17ce9686e5c0ac7f607d35f0a6712251a513d3532dbfdc WatchSource:0}: Error finding container 27a8882b8f73395b7c17ce9686e5c0ac7f607d35f0a6712251a513d3532dbfdc: Status 404 returned error can't find the container with id 27a8882b8f73395b7c17ce9686e5c0ac7f607d35f0a6712251a513d3532dbfdc Jul 07 12:50:33.189898 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:33.189859 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cts9s\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:33.191145 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.191116 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bfa2baa-f815-4056-84ac-53c5cd5f47c0" path="/var/lib/kubelet/pods/5bfa2baa-f815-4056-84ac-53c5cd5f47c0/volumes" Jul 07 12:50:33.191566 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.191542 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" path="/var/lib/kubelet/pods/5d7828c1-fb1f-44ef-9673-f6fda2ec4588/volumes" Jul 07 12:50:33.191982 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.191958 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84194c6c-2f33-4f7e-9439-1ab37bc29b85" path="/var/lib/kubelet/pods/84194c6c-2f33-4f7e-9439-1ab37bc29b85/volumes" Jul 07 12:50:33.192133 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:33.192103 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-ktrv7\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="84194c6c-2f33-4f7e-9439-1ab37bc29b85" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:33.194795 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:33.194764 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9947q\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="5d7828c1-fb1f-44ef-9673-f6fda2ec4588" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:33.547006 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.546966 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh" event={"ID":"9b049194-41bb-400f-ba08-9d205c54ea82","Type":"ContainerStarted","Data":"f10186cb779d6046d4ee7a5c28602a5444bda354f417003c157c0b702b038321"} Jul 07 12:50:33.547249 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.547025 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh" event={"ID":"9b049194-41bb-400f-ba08-9d205c54ea82","Type":"ContainerStarted","Data":"397389505d8f5dd534637b0ba9558e5f2457ef4516649dbf14e556d997e131b6"} Jul 07 12:50:33.547249 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.547053 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh" Jul 07 12:50:33.548309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.548289 2574 scope.go:117] "RemoveContainer" containerID="43461d8367e8de7ab05667b2872802c93ca0b65f79355d4c0fc1551d89cdc3db" Jul 07 12:50:33.548404 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.548290 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cts9s" Jul 07 12:50:33.550071 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.550042 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" event={"ID":"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6","Type":"ContainerStarted","Data":"a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0"} Jul 07 12:50:33.550185 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.550091 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" event={"ID":"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6","Type":"ContainerStarted","Data":"713ce3add73e96d927c785db7fd04af1838d588673fe1165a8926b3520efed84"} Jul 07 12:50:33.550292 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.550269 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:33.551646 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.551628 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9947q" Jul 07 12:50:33.553189 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.553166 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" event={"ID":"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f","Type":"ContainerStarted","Data":"c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215"} Jul 07 12:50:33.553293 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.553188 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:33.553293 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.553211 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" event={"ID":"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f","Type":"ContainerStarted","Data":"27a8882b8f73395b7c17ce9686e5c0ac7f607d35f0a6712251a513d3532dbfdc"} Jul 07 12:50:33.553408 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.553305 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 12:50:33.558902 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.558887 2574 scope.go:117] "RemoveContainer" containerID="c4b6efbb1a9eec48c5d4524e63bb0e7d74816f813d8d43d7469dad4871944903" Jul 07 12:50:33.576163 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.576094 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh" podStartSLOduration=1.576081777 podStartE2EDuration="1.576081777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:50:33.575403762 +0000 UTC m=+481.019446962" watchObservedRunningTime="2026-07-07 12:50:33.576081777 +0000 UTC m=+481.020124956" Jul 07 12:50:33.602193 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.602155 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" podStartSLOduration=1.6021415490000002 podStartE2EDuration="1.602141549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:50:33.600478145 +0000 UTC m=+481.044521355" watchObservedRunningTime="2026-07-07 12:50:33.602141549 +0000 UTC m=+481.046184730" Jul 07 12:50:33.633232 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:33.633173 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" podStartSLOduration=1.6331585290000001 podStartE2EDuration="1.633158529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:50:33.631768256 +0000 UTC m=+481.075811434" watchObservedRunningTime="2026-07-07 12:50:33.633158529 +0000 UTC m=+481.077201710" Jul 07 12:50:33.638172 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:33.638149 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-ktrv7\" is forbidden: User \"system:node:ip-10-0-135-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-212.ec2.internal' and this object" podUID="84194c6c-2f33-4f7e-9439-1ab37bc29b85" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ktrv7" Jul 07 12:50:37.189288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:37.189252 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:37.308602 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:37.308575 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:40.167878 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:40.167840 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:40.272079 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:40.272042 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:44.205039 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:44.205010 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:44.277786 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:44.277754 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:44.562366 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:44.562287 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dqxh" Jul 07 12:50:44.562366 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:44.562362 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 12:50:44.562549 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:44.562393 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:44.656798 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:44.656749 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz"] Jul 07 12:50:44.656991 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:44.656966 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" podUID="a6ef5723-7308-4d9a-9d1c-57fa4aed14b6" containerName="manager" containerID="cri-o://a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0" gracePeriod=10 Jul 07 12:50:44.902242 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:44.902222 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:45.052599 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.052575 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6xrv\" (UniqueName: \"kubernetes.io/projected/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-kube-api-access-g6xrv\") pod \"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6\" (UID: \"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6\") " Jul 07 12:50:45.052767 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.052680 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-extensions-socket-volume\") pod \"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6\" (UID: \"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6\") " Jul 07 12:50:45.053056 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.053029 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a6ef5723-7308-4d9a-9d1c-57fa4aed14b6" (UID: "a6ef5723-7308-4d9a-9d1c-57fa4aed14b6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:50:45.054556 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.054530 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-kube-api-access-g6xrv" (OuterVolumeSpecName: "kube-api-access-g6xrv") pod "a6ef5723-7308-4d9a-9d1c-57fa4aed14b6" (UID: "a6ef5723-7308-4d9a-9d1c-57fa4aed14b6"). InnerVolumeSpecName "kube-api-access-g6xrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:50:45.153986 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.153928 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-extensions-socket-volume\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:50:45.153986 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.153950 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6xrv\" (UniqueName: \"kubernetes.io/projected/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6-kube-api-access-g6xrv\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:50:45.618380 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.618342 2574 generic.go:356] "Generic (PLEG): container finished" podID="a6ef5723-7308-4d9a-9d1c-57fa4aed14b6" containerID="a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0" exitCode=0 Jul 07 12:50:45.618759 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.618403 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" Jul 07 12:50:45.618759 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.618427 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" event={"ID":"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6","Type":"ContainerDied","Data":"a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0"} Jul 07 12:50:45.618759 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.618504 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz" event={"ID":"a6ef5723-7308-4d9a-9d1c-57fa4aed14b6","Type":"ContainerDied","Data":"713ce3add73e96d927c785db7fd04af1838d588673fe1165a8926b3520efed84"} Jul 07 12:50:45.618759 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.618529 2574 scope.go:117] "RemoveContainer" containerID="a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0" Jul 07 12:50:45.627957 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.627922 2574 scope.go:117] "RemoveContainer" containerID="a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0" Jul 07 12:50:45.628336 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:50:45.628313 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0\": container with ID starting with a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0 not found: ID does not exist" containerID="a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0" Jul 07 12:50:45.628439 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.628340 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0"} err="failed to get container status \"a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0\": rpc error: code = NotFound desc = could not find container \"a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0\": container with ID starting with a933f9877ac18573ea85b30c57f482f22cff24efb86bb7e812fd441582fc4ec0 not found: ID does not exist" Jul 07 12:50:45.639004 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.638980 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz"] Jul 07 12:50:45.659228 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:45.659206 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8l6gz"] Jul 07 12:50:47.189774 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:47.189739 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6ef5723-7308-4d9a-9d1c-57fa4aed14b6" path="/var/lib/kubelet/pods/a6ef5723-7308-4d9a-9d1c-57fa4aed14b6/volumes" Jul 07 12:50:51.570037 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:51.570003 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:51.866074 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:51.866000 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:57.636017 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:57.635974 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:50:57.734444 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:50:57.734416 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:00.575212 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:00.575181 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:00.677517 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:00.677488 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:01.039007 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.038978 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t"] Jul 07 12:51:01.039388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.039374 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ef5723-7308-4d9a-9d1c-57fa4aed14b6" containerName="manager" Jul 07 12:51:01.039388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.039388 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ef5723-7308-4d9a-9d1c-57fa4aed14b6" containerName="manager" Jul 07 12:51:01.039519 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.039467 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6ef5723-7308-4d9a-9d1c-57fa4aed14b6" containerName="manager" Jul 07 12:51:01.041636 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.041619 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.044527 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.044508 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-hw2px\"" Jul 07 12:51:01.044739 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.044725 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 12:51:01.054868 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.054846 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t"] Jul 07 12:51:01.079067 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.079049 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:01.086348 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.086324 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.086457 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.086376 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.086457 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.086425 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.086566 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.086474 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.086566 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.086508 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.086566 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.086534 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.086759 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.086739 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.086822 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.086782 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkthx\" (UniqueName: \"kubernetes.io/projected/3e9c599b-ee72-4420-b164-d46c0e5e1049-kube-api-access-dkthx\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.086896 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.086871 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e9c599b-ee72-4420-b164-d46c0e5e1049-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.187578 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.187555 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.187733 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.187587 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.187733 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.187606 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.187733 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.187661 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.187733 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.187701 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkthx\" (UniqueName: \"kubernetes.io/projected/3e9c599b-ee72-4420-b164-d46c0e5e1049-kube-api-access-dkthx\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.187961 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.187837 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e9c599b-ee72-4420-b164-d46c0e5e1049-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.187961 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.187929 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.188076 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.187974 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.188076 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.188002 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.188182 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.188114 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.188391 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.188366 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.188391 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.188381 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.188540 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.188413 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.188600 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.188572 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e9c599b-ee72-4420-b164-d46c0e5e1049-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.190055 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.190035 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.190146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.190134 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.196626 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.196597 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e9c599b-ee72-4420-b164-d46c0e5e1049-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.196720 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.196712 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkthx\" (UniqueName: \"kubernetes.io/projected/3e9c599b-ee72-4420-b164-d46c0e5e1049-kube-api-access-dkthx\") pod \"maas-default-gateway-openshift-default-56654dcf7-qvw7t\" (UID: \"3e9c599b-ee72-4420-b164-d46c0e5e1049\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.220258 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.220236 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:01.352785 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.352730 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:01.492252 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.492228 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t"] Jul 07 12:51:01.494226 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:51:01.494200 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9c599b_ee72_4420_b164_d46c0e5e1049.slice/crio-13b4918905e532756fd7749d43321ea08ae249d201179fc0a1523927338a403c WatchSource:0}: Error finding container 13b4918905e532756fd7749d43321ea08ae249d201179fc0a1523927338a403c: Status 404 returned error can't find the container with id 13b4918905e532756fd7749d43321ea08ae249d201179fc0a1523927338a403c Jul 07 12:51:01.702273 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:01.702239 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" event={"ID":"3e9c599b-ee72-4420-b164-d46c0e5e1049","Type":"ContainerStarted","Data":"13b4918905e532756fd7749d43321ea08ae249d201179fc0a1523927338a403c"} Jul 07 12:51:03.902225 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:03.902185 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 12:51:03.902495 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:03.902265 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 12:51:03.902495 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:03.902297 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 12:51:04.720757 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:04.720725 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" event={"ID":"3e9c599b-ee72-4420-b164-d46c0e5e1049","Type":"ContainerStarted","Data":"e15912ca3bedb38958ff556184a556236cfc62a2c7b86abc6ea1fb30310b563a"} Jul 07 12:51:04.742985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:04.742942 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" podStartSLOduration=1.3371426579999999 podStartE2EDuration="3.742927652s" totalImagesPullingTime="2.405784994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:04.741911129 +0000 UTC m=+512.185954309" watchObservedRunningTime="2026-07-07 12:51:04.742927652 +0000 UTC m=+512.186970831" Jul 07 12:51:05.352906 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.352880 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:05.357091 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.357069 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:05.728264 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.728231 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:05.729285 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.729257 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:51:05.779254 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.779219 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5gjp7"] Jul 07 12:51:05.783004 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.782982 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:05.787995 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.787766 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qlvmx\"" Jul 07 12:51:05.788129 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.788070 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 12:51:05.797432 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.797404 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5gjp7"] Jul 07 12:51:05.828255 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.828218 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61e3a7c8-5157-4e79-a509-2e750f0645c8-config-file\") pod \"limitador-limitador-64c94fd64f-5gjp7\" (UID: \"61e3a7c8-5157-4e79-a509-2e750f0645c8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:05.828739 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.828714 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldght\" (UniqueName: \"kubernetes.io/projected/61e3a7c8-5157-4e79-a509-2e750f0645c8-kube-api-access-ldght\") pod \"limitador-limitador-64c94fd64f-5gjp7\" (UID: \"61e3a7c8-5157-4e79-a509-2e750f0645c8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:05.876602 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.876563 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5gjp7"] Jul 07 12:51:05.930151 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.930117 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldght\" (UniqueName: \"kubernetes.io/projected/61e3a7c8-5157-4e79-a509-2e750f0645c8-kube-api-access-ldght\") pod \"limitador-limitador-64c94fd64f-5gjp7\" (UID: \"61e3a7c8-5157-4e79-a509-2e750f0645c8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:05.930310 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.930155 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61e3a7c8-5157-4e79-a509-2e750f0645c8-config-file\") pod \"limitador-limitador-64c94fd64f-5gjp7\" (UID: \"61e3a7c8-5157-4e79-a509-2e750f0645c8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:05.930852 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.930828 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61e3a7c8-5157-4e79-a509-2e750f0645c8-config-file\") pod \"limitador-limitador-64c94fd64f-5gjp7\" (UID: \"61e3a7c8-5157-4e79-a509-2e750f0645c8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:05.940501 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.940474 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldght\" (UniqueName: \"kubernetes.io/projected/61e3a7c8-5157-4e79-a509-2e750f0645c8-kube-api-access-ldght\") pod \"limitador-limitador-64c94fd64f-5gjp7\" (UID: \"61e3a7c8-5157-4e79-a509-2e750f0645c8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:05.952391 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:05.952367 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:06.052744 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.052631 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:06.098817 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.098790 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:06.239204 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:51:06.239170 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61e3a7c8_5157_4e79_a509_2e750f0645c8.slice/crio-6302c5f014025be7e0b4776dd4e97b8fbdfca1dd5acab9b61ba06eb9daa6565a WatchSource:0}: Error finding container 6302c5f014025be7e0b4776dd4e97b8fbdfca1dd5acab9b61ba06eb9daa6565a: Status 404 returned error can't find the container with id 6302c5f014025be7e0b4776dd4e97b8fbdfca1dd5acab9b61ba06eb9daa6565a Jul 07 12:51:06.242446 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.242421 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5gjp7"] Jul 07 12:51:06.633952 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.633920 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:06.728303 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.728280 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:06.730735 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.730572 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-hvl8m"] Jul 07 12:51:06.736123 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.736099 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hvl8m" Jul 07 12:51:06.739144 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.739122 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5jxwj\"" Jul 07 12:51:06.740633 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.740609 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hvl8m"] Jul 07 12:51:06.741781 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.741752 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" event={"ID":"61e3a7c8-5157-4e79-a509-2e750f0645c8","Type":"ContainerStarted","Data":"6302c5f014025be7e0b4776dd4e97b8fbdfca1dd5acab9b61ba06eb9daa6565a"} Jul 07 12:51:06.839051 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.839023 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnlkn\" (UniqueName: \"kubernetes.io/projected/57c0be15-9def-47df-84b0-f73cb436cf41-kube-api-access-xnlkn\") pod \"authorino-75d556bd77-hvl8m\" (UID: \"57c0be15-9def-47df-84b0-f73cb436cf41\") " pod="kuadrant-system/authorino-75d556bd77-hvl8m" Jul 07 12:51:06.940411 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.940389 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnlkn\" (UniqueName: \"kubernetes.io/projected/57c0be15-9def-47df-84b0-f73cb436cf41-kube-api-access-xnlkn\") pod \"authorino-75d556bd77-hvl8m\" (UID: \"57c0be15-9def-47df-84b0-f73cb436cf41\") " pod="kuadrant-system/authorino-75d556bd77-hvl8m" Jul 07 12:51:06.949194 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:06.949166 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnlkn\" (UniqueName: \"kubernetes.io/projected/57c0be15-9def-47df-84b0-f73cb436cf41-kube-api-access-xnlkn\") pod \"authorino-75d556bd77-hvl8m\" (UID: \"57c0be15-9def-47df-84b0-f73cb436cf41\") " pod="kuadrant-system/authorino-75d556bd77-hvl8m" Jul 07 12:51:07.048305 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:07.048273 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hvl8m" Jul 07 12:51:07.229598 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:07.229510 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hvl8m"] Jul 07 12:51:07.231985 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:51:07.231954 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57c0be15_9def_47df_84b0_f73cb436cf41.slice/crio-87a4bb95096540b25eee5c71b4f1a48e6105b10571a948b008b63378dacb86b5 WatchSource:0}: Error finding container 87a4bb95096540b25eee5c71b4f1a48e6105b10571a948b008b63378dacb86b5: Status 404 returned error can't find the container with id 87a4bb95096540b25eee5c71b4f1a48e6105b10571a948b008b63378dacb86b5 Jul 07 12:51:07.752672 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:07.752597 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hvl8m" event={"ID":"57c0be15-9def-47df-84b0-f73cb436cf41","Type":"ContainerStarted","Data":"87a4bb95096540b25eee5c71b4f1a48e6105b10571a948b008b63378dacb86b5"} Jul 07 12:51:11.780562 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:11.780524 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" event={"ID":"61e3a7c8-5157-4e79-a509-2e750f0645c8","Type":"ContainerStarted","Data":"a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4"} Jul 07 12:51:11.781026 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:11.780712 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:11.782102 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:11.782078 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hvl8m" event={"ID":"57c0be15-9def-47df-84b0-f73cb436cf41","Type":"ContainerStarted","Data":"ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46"} Jul 07 12:51:11.799781 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:11.799714 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" podStartSLOduration=2.182162396 podStartE2EDuration="6.799684368s" totalImagesPullingTime="4.617521972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:11.796979602 +0000 UTC m=+519.241022800" watchObservedRunningTime="2026-07-07 12:51:11.799684368 +0000 UTC m=+519.243727547" Jul 07 12:51:11.812659 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:11.812620 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-hvl8m" podStartSLOduration=2.143647467 podStartE2EDuration="5.812608182s" totalImagesPullingTime="3.668960715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:11.812009202 +0000 UTC m=+519.256052382" watchObservedRunningTime="2026-07-07 12:51:11.812608182 +0000 UTC m=+519.256651360" Jul 07 12:51:22.789115 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:22.789086 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:23.024893 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.024837 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5gjp7"] Jul 07 12:51:23.025109 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.025054 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" podUID="61e3a7c8-5157-4e79-a509-2e750f0645c8" containerName="limitador" containerID="cri-o://a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4" gracePeriod=30 Jul 07 12:51:23.586213 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.586153 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:23.691635 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.691614 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61e3a7c8-5157-4e79-a509-2e750f0645c8-config-file\") pod \"61e3a7c8-5157-4e79-a509-2e750f0645c8\" (UID: \"61e3a7c8-5157-4e79-a509-2e750f0645c8\") " Jul 07 12:51:23.691777 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.691656 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldght\" (UniqueName: \"kubernetes.io/projected/61e3a7c8-5157-4e79-a509-2e750f0645c8-kube-api-access-ldght\") pod \"61e3a7c8-5157-4e79-a509-2e750f0645c8\" (UID: \"61e3a7c8-5157-4e79-a509-2e750f0645c8\") " Jul 07 12:51:23.691975 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.691943 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61e3a7c8-5157-4e79-a509-2e750f0645c8-config-file" (OuterVolumeSpecName: "config-file") pod "61e3a7c8-5157-4e79-a509-2e750f0645c8" (UID: "61e3a7c8-5157-4e79-a509-2e750f0645c8"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:51:23.693631 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.693609 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61e3a7c8-5157-4e79-a509-2e750f0645c8-kube-api-access-ldght" (OuterVolumeSpecName: "kube-api-access-ldght") pod "61e3a7c8-5157-4e79-a509-2e750f0645c8" (UID: "61e3a7c8-5157-4e79-a509-2e750f0645c8"). InnerVolumeSpecName "kube-api-access-ldght". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:51:23.793203 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.793179 2574 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61e3a7c8-5157-4e79-a509-2e750f0645c8-config-file\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:51:23.793203 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.793203 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldght\" (UniqueName: \"kubernetes.io/projected/61e3a7c8-5157-4e79-a509-2e750f0645c8-kube-api-access-ldght\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:51:23.852501 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.852435 2574 generic.go:356] "Generic (PLEG): container finished" podID="61e3a7c8-5157-4e79-a509-2e750f0645c8" containerID="a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4" exitCode=0 Jul 07 12:51:23.852501 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.852488 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" Jul 07 12:51:23.852662 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.852530 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" event={"ID":"61e3a7c8-5157-4e79-a509-2e750f0645c8","Type":"ContainerDied","Data":"a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4"} Jul 07 12:51:23.852662 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.852585 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5gjp7" event={"ID":"61e3a7c8-5157-4e79-a509-2e750f0645c8","Type":"ContainerDied","Data":"6302c5f014025be7e0b4776dd4e97b8fbdfca1dd5acab9b61ba06eb9daa6565a"} Jul 07 12:51:23.852662 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.852610 2574 scope.go:117] "RemoveContainer" containerID="a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4" Jul 07 12:51:23.862019 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.862002 2574 scope.go:117] "RemoveContainer" containerID="a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4" Jul 07 12:51:23.862262 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:51:23.862244 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4\": container with ID starting with a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4 not found: ID does not exist" containerID="a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4" Jul 07 12:51:23.862318 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.862268 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4"} err="failed to get container status \"a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4\": rpc error: code = NotFound desc = could not find container \"a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4\": container with ID starting with a503093b87afdda01f53f657e122d73b10369affdd3bea5ede85ea96378a86d4 not found: ID does not exist" Jul 07 12:51:23.878331 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.878297 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5gjp7"] Jul 07 12:51:23.883009 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:23.882987 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5gjp7"] Jul 07 12:51:25.189388 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:25.189316 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61e3a7c8-5157-4e79-a509-2e750f0645c8" path="/var/lib/kubelet/pods/61e3a7c8-5157-4e79-a509-2e750f0645c8/volumes" Jul 07 12:51:27.068866 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.068824 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:27.172000 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.171972 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:27.533569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.533535 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8"] Jul 07 12:51:27.533986 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.533972 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61e3a7c8-5157-4e79-a509-2e750f0645c8" containerName="limitador" Jul 07 12:51:27.534042 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.533988 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="61e3a7c8-5157-4e79-a509-2e750f0645c8" containerName="limitador" Jul 07 12:51:27.534091 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.534081 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="61e3a7c8-5157-4e79-a509-2e750f0645c8" containerName="limitador" Jul 07 12:51:27.536163 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.536143 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" Jul 07 12:51:27.539467 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.539446 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 07 12:51:27.540443 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.540423 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 07 12:51:27.540546 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.540460 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 07 12:51:27.540546 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.540482 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4bp5n\"" Jul 07 12:51:27.549069 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.549049 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8"] Jul 07 12:51:27.578352 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.578331 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:27.627039 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.627017 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln679\" (UniqueName: \"kubernetes.io/projected/4cc3de54-6448-4c74-97b5-4259711cc82d-kube-api-access-ln679\") pod \"postgres-69fb4594cf-fmxs8\" (UID: \"4cc3de54-6448-4c74-97b5-4259711cc82d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" Jul 07 12:51:27.627131 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.627087 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4cc3de54-6448-4c74-97b5-4259711cc82d-data\") pod \"postgres-69fb4594cf-fmxs8\" (UID: \"4cc3de54-6448-4c74-97b5-4259711cc82d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" Jul 07 12:51:27.728357 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.728334 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4cc3de54-6448-4c74-97b5-4259711cc82d-data\") pod \"postgres-69fb4594cf-fmxs8\" (UID: \"4cc3de54-6448-4c74-97b5-4259711cc82d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" Jul 07 12:51:27.728466 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.728396 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln679\" (UniqueName: \"kubernetes.io/projected/4cc3de54-6448-4c74-97b5-4259711cc82d-kube-api-access-ln679\") pod \"postgres-69fb4594cf-fmxs8\" (UID: \"4cc3de54-6448-4c74-97b5-4259711cc82d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" Jul 07 12:51:27.728730 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.728711 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4cc3de54-6448-4c74-97b5-4259711cc82d-data\") pod \"postgres-69fb4594cf-fmxs8\" (UID: \"4cc3de54-6448-4c74-97b5-4259711cc82d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" Jul 07 12:51:27.739051 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.739026 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln679\" (UniqueName: \"kubernetes.io/projected/4cc3de54-6448-4c74-97b5-4259711cc82d-kube-api-access-ln679\") pod \"postgres-69fb4594cf-fmxs8\" (UID: \"4cc3de54-6448-4c74-97b5-4259711cc82d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" Jul 07 12:51:27.848013 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.847945 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" Jul 07 12:51:27.985724 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:51:27.985453 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc3de54_6448_4c74_97b5_4259711cc82d.slice/crio-8fbf8a0c7759e44cdff47f86ef1b924ab041b27cdb480b4c53501d0a09095839 WatchSource:0}: Error finding container 8fbf8a0c7759e44cdff47f86ef1b924ab041b27cdb480b4c53501d0a09095839: Status 404 returned error can't find the container with id 8fbf8a0c7759e44cdff47f86ef1b924ab041b27cdb480b4c53501d0a09095839 Jul 07 12:51:27.989569 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:27.989528 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8"] Jul 07 12:51:28.884273 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:28.884193 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" event={"ID":"4cc3de54-6448-4c74-97b5-4259711cc82d","Type":"ContainerStarted","Data":"8fbf8a0c7759e44cdff47f86ef1b924ab041b27cdb480b4c53501d0a09095839"} Jul 07 12:51:32.908521 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:32.908433 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" event={"ID":"4cc3de54-6448-4c74-97b5-4259711cc82d","Type":"ContainerStarted","Data":"3f1d34478719b0e4b98cd787127d4fcc1f9908f70543675263c167a65a140725"} Jul 07 12:51:32.908959 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:32.908556 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" Jul 07 12:51:32.929371 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:32.929327 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" podStartSLOduration=1.331154983 podStartE2EDuration="5.929311721s" totalImagesPullingTime="4.598156738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:32.927346091 +0000 UTC m=+540.371389270" watchObservedRunningTime="2026-07-07 12:51:32.929311721 +0000 UTC m=+540.373354899" Jul 07 12:51:38.944123 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:38.944092 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fmxs8" Jul 07 12:51:39.290159 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:39.290086 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:39.720258 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:39.720230 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:41.164161 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.164132 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hvl8m"] Jul 07 12:51:41.164567 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.164337 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-hvl8m" podUID="57c0be15-9def-47df-84b0-f73cb436cf41" containerName="authorino" containerID="cri-o://ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46" gracePeriod=30 Jul 07 12:51:41.450221 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.450193 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hvl8m" Jul 07 12:51:41.561601 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.561561 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnlkn\" (UniqueName: \"kubernetes.io/projected/57c0be15-9def-47df-84b0-f73cb436cf41-kube-api-access-xnlkn\") pod \"57c0be15-9def-47df-84b0-f73cb436cf41\" (UID: \"57c0be15-9def-47df-84b0-f73cb436cf41\") " Jul 07 12:51:41.563544 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.563519 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57c0be15-9def-47df-84b0-f73cb436cf41-kube-api-access-xnlkn" (OuterVolumeSpecName: "kube-api-access-xnlkn") pod "57c0be15-9def-47df-84b0-f73cb436cf41" (UID: "57c0be15-9def-47df-84b0-f73cb436cf41"). InnerVolumeSpecName "kube-api-access-xnlkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:51:41.662953 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.662926 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnlkn\" (UniqueName: \"kubernetes.io/projected/57c0be15-9def-47df-84b0-f73cb436cf41-kube-api-access-xnlkn\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:51:41.957241 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.957214 2574 generic.go:356] "Generic (PLEG): container finished" podID="57c0be15-9def-47df-84b0-f73cb436cf41" containerID="ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46" exitCode=0 Jul 07 12:51:41.957399 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.957263 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hvl8m" Jul 07 12:51:41.957399 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.957302 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hvl8m" event={"ID":"57c0be15-9def-47df-84b0-f73cb436cf41","Type":"ContainerDied","Data":"ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46"} Jul 07 12:51:41.957399 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.957344 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hvl8m" event={"ID":"57c0be15-9def-47df-84b0-f73cb436cf41","Type":"ContainerDied","Data":"87a4bb95096540b25eee5c71b4f1a48e6105b10571a948b008b63378dacb86b5"} Jul 07 12:51:41.957399 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.957362 2574 scope.go:117] "RemoveContainer" containerID="ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46" Jul 07 12:51:41.966306 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.966291 2574 scope.go:117] "RemoveContainer" containerID="ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46" Jul 07 12:51:41.966535 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:51:41.966520 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46\": container with ID starting with ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46 not found: ID does not exist" containerID="ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46" Jul 07 12:51:41.966581 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.966543 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46"} err="failed to get container status \"ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46\": rpc error: code = NotFound desc = could not find container \"ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46\": container with ID starting with ecf9e0ee5a5d742d4bfbf89d81555513bb15f75f0bffcad9034111a741943a46 not found: ID does not exist" Jul 07 12:51:41.978170 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.978147 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hvl8m"] Jul 07 12:51:41.982310 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:41.982289 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hvl8m"] Jul 07 12:51:43.189647 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:43.189611 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57c0be15-9def-47df-84b0-f73cb436cf41" path="/var/lib/kubelet/pods/57c0be15-9def-47df-84b0-f73cb436cf41/volumes" Jul 07 12:51:45.599041 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.599005 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:45.625787 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.625756 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76f947c8bc-mwtc4"] Jul 07 12:51:45.626238 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.626222 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57c0be15-9def-47df-84b0-f73cb436cf41" containerName="authorino" Jul 07 12:51:45.626282 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.626241 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="57c0be15-9def-47df-84b0-f73cb436cf41" containerName="authorino" Jul 07 12:51:45.627708 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.626366 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="57c0be15-9def-47df-84b0-f73cb436cf41" containerName="authorino" Jul 07 12:51:45.630942 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.630921 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:51:45.633681 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.633660 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 12:51:45.634798 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.634778 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 12:51:45.634798 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.634790 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zntz8\"" Jul 07 12:51:45.643221 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.643200 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f947c8bc-mwtc4"] Jul 07 12:51:45.696967 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.696943 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-webhook-cert\") pod \"maas-controller-76f947c8bc-mwtc4\" (UID: \"2cdec4e9-9956-4e1d-916f-87b46fb51fc8\") " pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:51:45.697090 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.696978 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgjgn\" (UniqueName: \"kubernetes.io/projected/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-kube-api-access-dgjgn\") pod \"maas-controller-76f947c8bc-mwtc4\" (UID: \"2cdec4e9-9956-4e1d-916f-87b46fb51fc8\") " pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:51:45.704577 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.704541 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:45.770466 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.770422 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:45.798394 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.798374 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-webhook-cert\") pod \"maas-controller-76f947c8bc-mwtc4\" (UID: \"2cdec4e9-9956-4e1d-916f-87b46fb51fc8\") " pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:51:45.798496 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.798405 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgjgn\" (UniqueName: \"kubernetes.io/projected/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-kube-api-access-dgjgn\") pod \"maas-controller-76f947c8bc-mwtc4\" (UID: \"2cdec4e9-9956-4e1d-916f-87b46fb51fc8\") " pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:51:45.800747 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.800731 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-webhook-cert\") pod \"maas-controller-76f947c8bc-mwtc4\" (UID: \"2cdec4e9-9956-4e1d-916f-87b46fb51fc8\") " pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:51:45.807601 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.807577 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgjgn\" (UniqueName: \"kubernetes.io/projected/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-kube-api-access-dgjgn\") pod \"maas-controller-76f947c8bc-mwtc4\" (UID: \"2cdec4e9-9956-4e1d-916f-87b46fb51fc8\") " pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:51:45.812812 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.812789 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-557d84d488-5gmf7"] Jul 07 12:51:45.815895 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.815877 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:51:45.827290 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.827270 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-557d84d488-5gmf7"] Jul 07 12:51:45.899117 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.899041 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/088cf233-cbc9-45fb-b79a-cc5f1febf58d-webhook-cert\") pod \"maas-controller-557d84d488-5gmf7\" (UID: \"088cf233-cbc9-45fb-b79a-cc5f1febf58d\") " pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:51:45.899117 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.899094 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgbz7\" (UniqueName: \"kubernetes.io/projected/088cf233-cbc9-45fb-b79a-cc5f1febf58d-kube-api-access-mgbz7\") pod \"maas-controller-557d84d488-5gmf7\" (UID: \"088cf233-cbc9-45fb-b79a-cc5f1febf58d\") " pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:51:45.942605 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.942580 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:51:45.999730 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.999673 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/088cf233-cbc9-45fb-b79a-cc5f1febf58d-webhook-cert\") pod \"maas-controller-557d84d488-5gmf7\" (UID: \"088cf233-cbc9-45fb-b79a-cc5f1febf58d\") " pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:51:45.999853 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:45.999815 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgbz7\" (UniqueName: \"kubernetes.io/projected/088cf233-cbc9-45fb-b79a-cc5f1febf58d-kube-api-access-mgbz7\") pod \"maas-controller-557d84d488-5gmf7\" (UID: \"088cf233-cbc9-45fb-b79a-cc5f1febf58d\") " pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:51:46.002504 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:46.002474 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/088cf233-cbc9-45fb-b79a-cc5f1febf58d-webhook-cert\") pod \"maas-controller-557d84d488-5gmf7\" (UID: \"088cf233-cbc9-45fb-b79a-cc5f1febf58d\") " pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:51:46.015710 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:46.015676 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgbz7\" (UniqueName: \"kubernetes.io/projected/088cf233-cbc9-45fb-b79a-cc5f1febf58d-kube-api-access-mgbz7\") pod \"maas-controller-557d84d488-5gmf7\" (UID: \"088cf233-cbc9-45fb-b79a-cc5f1febf58d\") " pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:51:46.075423 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:46.075400 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f947c8bc-mwtc4"] Jul 07 12:51:46.076641 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:51:46.076617 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cdec4e9_9956_4e1d_916f_87b46fb51fc8.slice/crio-ca3c3deb88fc8cc48f112c607a1d83fe36ad24b6fa2a94ce5d2c076120f1698e WatchSource:0}: Error finding container ca3c3deb88fc8cc48f112c607a1d83fe36ad24b6fa2a94ce5d2c076120f1698e: Status 404 returned error can't find the container with id ca3c3deb88fc8cc48f112c607a1d83fe36ad24b6fa2a94ce5d2c076120f1698e Jul 07 12:51:46.126354 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:46.126328 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:51:46.261184 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:46.261158 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-557d84d488-5gmf7"] Jul 07 12:51:46.262956 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:51:46.262935 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088cf233_cbc9_45fb_b79a_cc5f1febf58d.slice/crio-f78537c11d011fd36584c867c9891d60ad1d202fd31213614dd9e8e8390c9c75 WatchSource:0}: Error finding container f78537c11d011fd36584c867c9891d60ad1d202fd31213614dd9e8e8390c9c75: Status 404 returned error can't find the container with id f78537c11d011fd36584c867c9891d60ad1d202fd31213614dd9e8e8390c9c75 Jul 07 12:51:46.988098 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:46.988059 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-557d84d488-5gmf7" event={"ID":"088cf233-cbc9-45fb-b79a-cc5f1febf58d","Type":"ContainerStarted","Data":"f78537c11d011fd36584c867c9891d60ad1d202fd31213614dd9e8e8390c9c75"} Jul 07 12:51:46.990145 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:46.990114 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" event={"ID":"2cdec4e9-9956-4e1d-916f-87b46fb51fc8","Type":"ContainerStarted","Data":"ca3c3deb88fc8cc48f112c607a1d83fe36ad24b6fa2a94ce5d2c076120f1698e"} Jul 07 12:51:48.389316 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:48.389282 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:48.578593 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:48.578562 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:51:49.004824 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:49.004794 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-557d84d488-5gmf7" event={"ID":"088cf233-cbc9-45fb-b79a-cc5f1febf58d","Type":"ContainerStarted","Data":"ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8"} Jul 07 12:51:49.004998 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:49.004863 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:51:49.006436 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:49.006408 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" event={"ID":"2cdec4e9-9956-4e1d-916f-87b46fb51fc8","Type":"ContainerStarted","Data":"a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130"} Jul 07 12:51:49.006579 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:49.006490 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:51:49.030276 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:49.030230 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-557d84d488-5gmf7" podStartSLOduration=2.139829427 podStartE2EDuration="4.030214948s" totalImagesPullingTime="1.890385521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:49.028111127 +0000 UTC m=+556.472154337" watchObservedRunningTime="2026-07-07 12:51:49.030214948 +0000 UTC m=+556.474258106" Jul 07 12:51:49.058243 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:51:49.058200 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" podStartSLOduration=1.971282435 podStartE2EDuration="4.058163735s" totalImagesPullingTime="2.0868813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:49.055422633 +0000 UTC m=+556.499465832" watchObservedRunningTime="2026-07-07 12:51:49.058163735 +0000 UTC m=+556.502206914" Jul 07 12:52:00.018460 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:00.018428 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:52:00.019091 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:00.019073 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:52:00.096212 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:00.096174 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f947c8bc-mwtc4"] Jul 07 12:52:00.096415 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:00.096385 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" podUID="2cdec4e9-9956-4e1d-916f-87b46fb51fc8" containerName="manager" containerID="cri-o://a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130" gracePeriod=30 Jul 07 12:52:00.345513 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:00.345492 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:52:00.440905 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:00.440878 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-webhook-cert\") pod \"2cdec4e9-9956-4e1d-916f-87b46fb51fc8\" (UID: \"2cdec4e9-9956-4e1d-916f-87b46fb51fc8\") " Jul 07 12:52:00.441113 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:00.441004 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgjgn\" (UniqueName: \"kubernetes.io/projected/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-kube-api-access-dgjgn\") pod \"2cdec4e9-9956-4e1d-916f-87b46fb51fc8\" (UID: \"2cdec4e9-9956-4e1d-916f-87b46fb51fc8\") " Jul 07 12:52:00.442988 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:00.442952 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2cdec4e9-9956-4e1d-916f-87b46fb51fc8" (UID: "2cdec4e9-9956-4e1d-916f-87b46fb51fc8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:52:00.443117 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:00.442997 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-kube-api-access-dgjgn" (OuterVolumeSpecName: "kube-api-access-dgjgn") pod "2cdec4e9-9956-4e1d-916f-87b46fb51fc8" (UID: "2cdec4e9-9956-4e1d-916f-87b46fb51fc8"). InnerVolumeSpecName "kube-api-access-dgjgn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:52:00.542072 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:00.542046 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgjgn\" (UniqueName: \"kubernetes.io/projected/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-kube-api-access-dgjgn\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:52:00.542072 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:00.542071 2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cdec4e9-9956-4e1d-916f-87b46fb51fc8-webhook-cert\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:52:01.074446 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:01.074415 2574 generic.go:356] "Generic (PLEG): container finished" podID="2cdec4e9-9956-4e1d-916f-87b46fb51fc8" containerID="a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130" exitCode=0 Jul 07 12:52:01.074831 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:01.074455 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" Jul 07 12:52:01.074831 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:01.074505 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" event={"ID":"2cdec4e9-9956-4e1d-916f-87b46fb51fc8","Type":"ContainerDied","Data":"a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130"} Jul 07 12:52:01.074831 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:01.074551 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f947c8bc-mwtc4" event={"ID":"2cdec4e9-9956-4e1d-916f-87b46fb51fc8","Type":"ContainerDied","Data":"ca3c3deb88fc8cc48f112c607a1d83fe36ad24b6fa2a94ce5d2c076120f1698e"} Jul 07 12:52:01.074831 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:01.074574 2574 scope.go:117] "RemoveContainer" containerID="a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130" Jul 07 12:52:01.083985 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:01.083965 2574 scope.go:117] "RemoveContainer" containerID="a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130" Jul 07 12:52:01.084227 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:52:01.084210 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130\": container with ID starting with a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130 not found: ID does not exist" containerID="a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130" Jul 07 12:52:01.084268 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:01.084236 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130"} err="failed to get container status \"a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130\": rpc error: code = NotFound desc = could not find container \"a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130\": container with ID starting with a3a4a24a8d65656d4be3b06ba14bba548b3afc25dab28e2ca4a919a580bc5130 not found: ID does not exist" Jul 07 12:52:01.096504 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:01.096483 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f947c8bc-mwtc4"] Jul 07 12:52:01.100540 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:01.100516 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76f947c8bc-mwtc4"] Jul 07 12:52:01.188806 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:01.188783 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cdec4e9-9956-4e1d-916f-87b46fb51fc8" path="/var/lib/kubelet/pods/2cdec4e9-9956-4e1d-916f-87b46fb51fc8/volumes" Jul 07 12:52:18.021837 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.021787 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:52:18.758765 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.758726 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs"] Jul 07 12:52:18.759193 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.759178 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cdec4e9-9956-4e1d-916f-87b46fb51fc8" containerName="manager" Jul 07 12:52:18.759240 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.759195 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cdec4e9-9956-4e1d-916f-87b46fb51fc8" containerName="manager" Jul 07 12:52:18.759293 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.759284 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cdec4e9-9956-4e1d-916f-87b46fb51fc8" containerName="manager" Jul 07 12:52:18.779230 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.779201 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs"] Jul 07 12:52:18.779403 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.779286 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:18.788062 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.788033 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 07 12:52:18.788214 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.788079 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fwnbg\"" Jul 07 12:52:18.809559 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.809499 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d52ada50-1e28-4449-9822-eb79c2f8e140-maas-api-tls\") pod \"maas-api-89bf6fb6d-cpkqs\" (UID: \"d52ada50-1e28-4449-9822-eb79c2f8e140\") " pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:18.809775 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.809660 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t87sz\" (UniqueName: \"kubernetes.io/projected/d52ada50-1e28-4449-9822-eb79c2f8e140-kube-api-access-t87sz\") pod \"maas-api-89bf6fb6d-cpkqs\" (UID: \"d52ada50-1e28-4449-9822-eb79c2f8e140\") " pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:18.809775 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.809735 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d52ada50-1e28-4449-9822-eb79c2f8e140-openshift-service-ca\") pod \"maas-api-89bf6fb6d-cpkqs\" (UID: \"d52ada50-1e28-4449-9822-eb79c2f8e140\") " pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:18.911104 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.911066 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t87sz\" (UniqueName: \"kubernetes.io/projected/d52ada50-1e28-4449-9822-eb79c2f8e140-kube-api-access-t87sz\") pod \"maas-api-89bf6fb6d-cpkqs\" (UID: \"d52ada50-1e28-4449-9822-eb79c2f8e140\") " pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:18.911301 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.911118 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d52ada50-1e28-4449-9822-eb79c2f8e140-openshift-service-ca\") pod \"maas-api-89bf6fb6d-cpkqs\" (UID: \"d52ada50-1e28-4449-9822-eb79c2f8e140\") " pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:18.911301 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.911163 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d52ada50-1e28-4449-9822-eb79c2f8e140-maas-api-tls\") pod \"maas-api-89bf6fb6d-cpkqs\" (UID: \"d52ada50-1e28-4449-9822-eb79c2f8e140\") " pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:18.911888 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.911864 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d52ada50-1e28-4449-9822-eb79c2f8e140-openshift-service-ca\") pod \"maas-api-89bf6fb6d-cpkqs\" (UID: \"d52ada50-1e28-4449-9822-eb79c2f8e140\") " pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:18.913486 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.913470 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d52ada50-1e28-4449-9822-eb79c2f8e140-maas-api-tls\") pod \"maas-api-89bf6fb6d-cpkqs\" (UID: \"d52ada50-1e28-4449-9822-eb79c2f8e140\") " pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:18.925948 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:18.925926 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t87sz\" (UniqueName: \"kubernetes.io/projected/d52ada50-1e28-4449-9822-eb79c2f8e140-kube-api-access-t87sz\") pod \"maas-api-89bf6fb6d-cpkqs\" (UID: \"d52ada50-1e28-4449-9822-eb79c2f8e140\") " pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:19.093404 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.093315 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:19.240464 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.240436 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs"] Jul 07 12:52:19.242460 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:52:19.242436 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52ada50_1e28_4449_9822_eb79c2f8e140.slice/crio-68503c5f8f3af699283761c615bb31ffb5ec3a88b18f5eb119a86a6649edbccc WatchSource:0}: Error finding container 68503c5f8f3af699283761c615bb31ffb5ec3a88b18f5eb119a86a6649edbccc: Status 404 returned error can't find the container with id 68503c5f8f3af699283761c615bb31ffb5ec3a88b18f5eb119a86a6649edbccc Jul 07 12:52:19.312263 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.312222 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:52:19.408108 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.408002 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:52:19.453159 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.453130 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-7dmtd"] Jul 07 12:52:19.457882 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.457862 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" Jul 07 12:52:19.460732 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.460712 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 12:52:19.470497 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.470478 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-7dmtd"] Jul 07 12:52:19.494933 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.494906 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:52:19.515572 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.515547 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29d948e-020c-41f3-a03e-ddd60aea93cc-plugins-config-volume\") pod \"payload-processing-58f86c7798-7dmtd\" (UID: \"e29d948e-020c-41f3-a03e-ddd60aea93cc\") " pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" Jul 07 12:52:19.515665 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.515589 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5snb4\" (UniqueName: \"kubernetes.io/projected/e29d948e-020c-41f3-a03e-ddd60aea93cc-kube-api-access-5snb4\") pod \"payload-processing-58f86c7798-7dmtd\" (UID: \"e29d948e-020c-41f3-a03e-ddd60aea93cc\") " pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" Jul 07 12:52:19.527142 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.527123 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-jw8hs"] Jul 07 12:52:19.530727 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.530683 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" Jul 07 12:52:19.535800 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.535784 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4fqxn\"" Jul 07 12:52:19.542701 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.542669 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-jw8hs"] Jul 07 12:52:19.617215 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.617184 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d35dfec3-b630-4978-95a2-7276835fb5d8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-jw8hs\" (UID: \"d35dfec3-b630-4978-95a2-7276835fb5d8\") " pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" Jul 07 12:52:19.617402 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.617251 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kzj2\" (UniqueName: \"kubernetes.io/projected/d35dfec3-b630-4978-95a2-7276835fb5d8-kube-api-access-2kzj2\") pod \"payload-pre-processing-7964457696-jw8hs\" (UID: \"d35dfec3-b630-4978-95a2-7276835fb5d8\") " pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" Jul 07 12:52:19.617402 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.617317 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29d948e-020c-41f3-a03e-ddd60aea93cc-plugins-config-volume\") pod \"payload-processing-58f86c7798-7dmtd\" (UID: \"e29d948e-020c-41f3-a03e-ddd60aea93cc\") " pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" Jul 07 12:52:19.617402 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.617345 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5snb4\" (UniqueName: \"kubernetes.io/projected/e29d948e-020c-41f3-a03e-ddd60aea93cc-kube-api-access-5snb4\") pod \"payload-processing-58f86c7798-7dmtd\" (UID: \"e29d948e-020c-41f3-a03e-ddd60aea93cc\") " pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" Jul 07 12:52:19.618026 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.618001 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29d948e-020c-41f3-a03e-ddd60aea93cc-plugins-config-volume\") pod \"payload-processing-58f86c7798-7dmtd\" (UID: \"e29d948e-020c-41f3-a03e-ddd60aea93cc\") " pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" Jul 07 12:52:19.626567 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.626532 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5snb4\" (UniqueName: \"kubernetes.io/projected/e29d948e-020c-41f3-a03e-ddd60aea93cc-kube-api-access-5snb4\") pod \"payload-processing-58f86c7798-7dmtd\" (UID: \"e29d948e-020c-41f3-a03e-ddd60aea93cc\") " pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" Jul 07 12:52:19.718319 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.718294 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d35dfec3-b630-4978-95a2-7276835fb5d8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-jw8hs\" (UID: \"d35dfec3-b630-4978-95a2-7276835fb5d8\") " pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" Jul 07 12:52:19.718462 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.718390 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kzj2\" (UniqueName: \"kubernetes.io/projected/d35dfec3-b630-4978-95a2-7276835fb5d8-kube-api-access-2kzj2\") pod \"payload-pre-processing-7964457696-jw8hs\" (UID: \"d35dfec3-b630-4978-95a2-7276835fb5d8\") " pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" Jul 07 12:52:19.719087 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.719067 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d35dfec3-b630-4978-95a2-7276835fb5d8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-jw8hs\" (UID: \"d35dfec3-b630-4978-95a2-7276835fb5d8\") " pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" Jul 07 12:52:19.727584 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.727564 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kzj2\" (UniqueName: \"kubernetes.io/projected/d35dfec3-b630-4978-95a2-7276835fb5d8-kube-api-access-2kzj2\") pod \"payload-pre-processing-7964457696-jw8hs\" (UID: \"d35dfec3-b630-4978-95a2-7276835fb5d8\") " pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" Jul 07 12:52:19.769710 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.769666 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" Jul 07 12:52:19.841059 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.841033 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" Jul 07 12:52:19.918977 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.918953 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-7dmtd"] Jul 07 12:52:19.921040 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:52:19.921014 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode29d948e_020c_41f3_a03e_ddd60aea93cc.slice/crio-bf869167373e5ded6871a573b1e4802fb4ff52812ae5d626216318388e204f4f WatchSource:0}: Error finding container bf869167373e5ded6871a573b1e4802fb4ff52812ae5d626216318388e204f4f: Status 404 returned error can't find the container with id bf869167373e5ded6871a573b1e4802fb4ff52812ae5d626216318388e204f4f Jul 07 12:52:19.992277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:19.992251 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-jw8hs"] Jul 07 12:52:19.994479 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:52:19.994446 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd35dfec3_b630_4978_95a2_7276835fb5d8.slice/crio-5af5352f3d6aa7167fd7363f43ad888778bd537dfa572a009fea68d9f183674f WatchSource:0}: Error finding container 5af5352f3d6aa7167fd7363f43ad888778bd537dfa572a009fea68d9f183674f: Status 404 returned error can't find the container with id 5af5352f3d6aa7167fd7363f43ad888778bd537dfa572a009fea68d9f183674f Jul 07 12:52:20.175138 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:20.175092 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" event={"ID":"e29d948e-020c-41f3-a03e-ddd60aea93cc","Type":"ContainerStarted","Data":"bf869167373e5ded6871a573b1e4802fb4ff52812ae5d626216318388e204f4f"} Jul 07 12:52:20.176683 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:20.176651 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" event={"ID":"d35dfec3-b630-4978-95a2-7276835fb5d8","Type":"ContainerStarted","Data":"5af5352f3d6aa7167fd7363f43ad888778bd537dfa572a009fea68d9f183674f"} Jul 07 12:52:20.178075 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:20.178041 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" event={"ID":"d52ada50-1e28-4449-9822-eb79c2f8e140","Type":"ContainerStarted","Data":"68503c5f8f3af699283761c615bb31ffb5ec3a88b18f5eb119a86a6649edbccc"} Jul 07 12:52:22.193938 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:22.193903 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" event={"ID":"d52ada50-1e28-4449-9822-eb79c2f8e140","Type":"ContainerStarted","Data":"3523949def67b1dd210e6e747707a375bf96c95e619e4f672e2300ef720df328"} Jul 07 12:52:22.194373 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:22.193972 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:22.195452 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:22.195424 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" event={"ID":"e29d948e-020c-41f3-a03e-ddd60aea93cc","Type":"ContainerStarted","Data":"1758b5466be80ddf7414c70c6c3d24c69d9c088b3f5e9798af6fec3c6db30c9e"} Jul 07 12:52:22.195594 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:22.195479 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" Jul 07 12:52:22.196968 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:22.196941 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" event={"ID":"d35dfec3-b630-4978-95a2-7276835fb5d8","Type":"ContainerStarted","Data":"271d70ccd5f9f211117d7391a86597244d900325d85b51296e3dbc1697b96470"} Jul 07 12:52:22.197087 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:22.196999 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" Jul 07 12:52:22.211722 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:22.211635 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" podStartSLOduration=1.767384531 podStartE2EDuration="4.211601406s" totalImagesPullingTime="2.444216875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:52:22.210335802 +0000 UTC m=+589.654378981" watchObservedRunningTime="2026-07-07 12:52:22.211601406 +0000 UTC m=+589.655644586" Jul 07 12:52:22.230048 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:22.229993 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" podStartSLOduration=1.464898327 podStartE2EDuration="3.229974123s" totalImagesPullingTime="1.765075796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:52:22.226258967 +0000 UTC m=+589.670302172" watchObservedRunningTime="2026-07-07 12:52:22.229974123 +0000 UTC m=+589.674017304" Jul 07 12:52:22.244523 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:22.242048 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" podStartSLOduration=1.5485159450000001 podStartE2EDuration="3.242030214s" totalImagesPullingTime="1.693514269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:52:22.24167656 +0000 UTC m=+589.685719742" watchObservedRunningTime="2026-07-07 12:52:22.242030214 +0000 UTC m=+589.686073396" Jul 07 12:52:28.207412 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:28.207383 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-89bf6fb6d-cpkqs" Jul 07 12:52:28.232670 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:28.232635 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e9c599b-ee72-4420-b164-d46c0e5e1049" containerID="e15912ca3bedb38958ff556184a556236cfc62a2c7b86abc6ea1fb30310b563a" exitCode=0 Jul 07 12:52:28.232824 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:28.232712 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" event={"ID":"3e9c599b-ee72-4420-b164-d46c0e5e1049","Type":"ContainerDied","Data":"e15912ca3bedb38958ff556184a556236cfc62a2c7b86abc6ea1fb30310b563a"} Jul 07 12:52:28.233077 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:28.233061 2574 scope.go:117] "RemoveContainer" containerID="e15912ca3bedb38958ff556184a556236cfc62a2c7b86abc6ea1fb30310b563a" Jul 07 12:52:28.234209 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:28.234174 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 12:52:28.234288 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:28.234261 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 12:52:28.234332 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:28.234319 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 12:52:29.227594 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:29.227561 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:52:29.241499 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:29.241461 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" event={"ID":"3e9c599b-ee72-4420-b164-d46c0e5e1049","Type":"ContainerStarted","Data":"09a351447210f188cacc64f525a7b1c4691fdff15c330171b74ed29ee4662ee6"} Jul 07 12:52:29.314278 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:29.314243 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:52:29.353053 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:29.353021 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:52:29.353220 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:29.353203 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qvw7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 07 12:52:29.353277 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:29.353258 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" podUID="3e9c599b-ee72-4420-b164-d46c0e5e1049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 07 12:52:30.353608 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:30.353574 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qvw7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 07 12:52:30.354011 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:30.353630 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" podUID="3e9c599b-ee72-4420-b164-d46c0e5e1049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 07 12:52:31.353680 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:31.353646 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qvw7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 07 12:52:31.354087 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:31.353727 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" podUID="3e9c599b-ee72-4420-b164-d46c0e5e1049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 07 12:52:32.353162 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:32.353125 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qvw7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 07 12:52:32.353325 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:32.353190 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" podUID="3e9c599b-ee72-4420-b164-d46c0e5e1049" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 07 12:52:33.204190 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:33.204162 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-7dmtd" Jul 07 12:52:33.204534 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:33.204357 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-jw8hs" Jul 07 12:52:33.357478 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:33.357409 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:52:33.357728 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:33.357682 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:52:33.358450 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:33.358432 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qvw7t" Jul 07 12:52:34.627005 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:34.626970 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:52:49.372705 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.372646 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn"] Jul 07 12:52:49.376900 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.376877 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.380906 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.380886 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f7d92\"" Jul 07 12:52:49.381143 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.380904 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 12:52:49.381252 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.380952 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 12:52:49.381314 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.380978 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 12:52:49.389921 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.389891 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn"] Jul 07 12:52:49.428499 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.428473 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:52:49.529202 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.529171 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.529323 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.529216 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.529376 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.529331 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25cg4\" (UniqueName: \"kubernetes.io/projected/e7b2a822-1e46-49ff-8917-71654365e360-kube-api-access-25cg4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.529424 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.529381 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.529474 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.529429 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.529474 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.529453 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7b2a822-1e46-49ff-8917-71654365e360-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.630264 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.630193 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.630264 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.630226 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.630264 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.630248 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7b2a822-1e46-49ff-8917-71654365e360-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.630501 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.630295 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.630501 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.630320 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.630501 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.630387 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25cg4\" (UniqueName: \"kubernetes.io/projected/e7b2a822-1e46-49ff-8917-71654365e360-kube-api-access-25cg4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.630666 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.630595 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.630774 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.630756 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.630829 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.630768 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.632342 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.632322 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7b2a822-1e46-49ff-8917-71654365e360-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.632613 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.632593 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7b2a822-1e46-49ff-8917-71654365e360-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.639820 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.639800 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25cg4\" (UniqueName: \"kubernetes.io/projected/e7b2a822-1e46-49ff-8917-71654365e360-kube-api-access-25cg4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hn8jn\" (UID: \"e7b2a822-1e46-49ff-8917-71654365e360\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.691361 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.691336 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:49.831566 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.831533 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn"] Jul 07 12:52:49.832297 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:52:49.832270 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b2a822_1e46_49ff_8917_71654365e360.slice/crio-84a7d7f558a794a1ed75e2e2b8b304566d5d40b6f1ae0de62d1a050c2159f87a WatchSource:0}: Error finding container 84a7d7f558a794a1ed75e2e2b8b304566d5d40b6f1ae0de62d1a050c2159f87a: Status 404 returned error can't find the container with id 84a7d7f558a794a1ed75e2e2b8b304566d5d40b6f1ae0de62d1a050c2159f87a Jul 07 12:52:49.833533 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:49.833515 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 12:52:50.356869 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:50.356834 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" event={"ID":"e7b2a822-1e46-49ff-8917-71654365e360","Type":"ContainerStarted","Data":"84a7d7f558a794a1ed75e2e2b8b304566d5d40b6f1ae0de62d1a050c2159f87a"} Jul 07 12:52:51.363084 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:51.363042 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" event={"ID":"e7b2a822-1e46-49ff-8917-71654365e360","Type":"ContainerStarted","Data":"ec422733fbcc614c97d2e1a4b5db9645212f7f6f135ba2091d1fd51f888ccb85"} Jul 07 12:52:51.384641 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:51.384596 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" podStartSLOduration=1.326523771 podStartE2EDuration="2.384581835s" totalImagesPullingTime="1.058058064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:52:51.382048402 +0000 UTC m=+618.826091583" watchObservedRunningTime="2026-07-07 12:52:51.384581835 +0000 UTC m=+618.828625015" Jul 07 12:52:56.075478 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.075391 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2"] Jul 07 12:52:56.083305 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.083156 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.088311 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.088287 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 12:52:56.092146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.092103 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.092146 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.092143 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.092356 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.092185 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.092356 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.092253 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m26d6\" (UniqueName: \"kubernetes.io/projected/60121699-e818-4fa6-8277-ce2ada9a7acf-kube-api-access-m26d6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.092356 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.092281 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60121699-e818-4fa6-8277-ce2ada9a7acf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.092356 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.092305 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.093513 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.093491 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2"] Jul 07 12:52:56.146008 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.145982 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:52:56.193627 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.193600 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.193784 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.193632 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.193784 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.193667 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.193784 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.193765 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m26d6\" (UniqueName: \"kubernetes.io/projected/60121699-e818-4fa6-8277-ce2ada9a7acf-kube-api-access-m26d6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.193981 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.193794 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60121699-e818-4fa6-8277-ce2ada9a7acf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.193981 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.193818 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.194113 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.194034 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.194113 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.194100 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.194262 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.194109 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.196047 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.196019 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60121699-e818-4fa6-8277-ce2ada9a7acf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.196240 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.196224 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60121699-e818-4fa6-8277-ce2ada9a7acf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.203265 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.203248 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m26d6\" (UniqueName: \"kubernetes.io/projected/60121699-e818-4fa6-8277-ce2ada9a7acf-kube-api-access-m26d6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jx4q2\" (UID: \"60121699-e818-4fa6-8277-ce2ada9a7acf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.396050 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.395983 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:52:56.535922 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:56.535899 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2"] Jul 07 12:52:56.538016 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:52:56.537988 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60121699_e818_4fa6_8277_ce2ada9a7acf.slice/crio-1f3242cd482eb0655c53093e3a5c2037c89935165290f0b957860fe6fad9e0d0 WatchSource:0}: Error finding container 1f3242cd482eb0655c53093e3a5c2037c89935165290f0b957860fe6fad9e0d0: Status 404 returned error can't find the container with id 1f3242cd482eb0655c53093e3a5c2037c89935165290f0b957860fe6fad9e0d0 Jul 07 12:52:57.396069 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:57.396035 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" event={"ID":"60121699-e818-4fa6-8277-ce2ada9a7acf","Type":"ContainerStarted","Data":"7091e47fecb87cd964707a0d5feb1352b16bbfedc91d8b10fe62a9858aff9169"} Jul 07 12:52:57.396420 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:57.396085 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" event={"ID":"60121699-e818-4fa6-8277-ce2ada9a7acf","Type":"ContainerStarted","Data":"1f3242cd482eb0655c53093e3a5c2037c89935165290f0b957860fe6fad9e0d0"} Jul 07 12:52:57.439973 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:57.439932 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" podStartSLOduration=1.264744508 podStartE2EDuration="1.439918515s" totalImagesPullingTime="175.174007ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:52:57.437829106 +0000 UTC m=+624.881872303" watchObservedRunningTime="2026-07-07 12:52:57.439918515 +0000 UTC m=+624.883961693" Jul 07 12:52:59.499061 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:59.499016 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:52:59.691767 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:59.691735 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:59.691916 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:59.691798 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:59.704028 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:59.704000 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:52:59.917896 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:52:59.917817 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:53:00.425263 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:53:00.425232 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hn8jn" Jul 07 12:53:06.397028 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:53:06.396994 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:53:06.397473 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:53:06.397051 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:53:06.411404 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:53:06.411377 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:53:06.457751 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:53:06.457729 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jx4q2" Jul 07 12:53:08.948118 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:53:08.948078 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:53:21.009701 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:53:21.009660 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:53:54.082606 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:53:54.082575 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:53:56.027187 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:53:56.027158 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:54:26.477156 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:54:26.477072 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:54:26.575929 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:54:26.575902 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:54:33.298443 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:54:33.298412 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:54:33.425745 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:54:33.425714 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:54:34.490052 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:54:34.490002 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:54:34.589064 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:54:34.589031 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:55:01.100764 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:01.100730 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:55:01.200857 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:01.200821 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:55:54.974971 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:54.974899 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:55:55.079650 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.079605 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-557d84d488-5gmf7"] Jul 07 12:55:55.079924 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.079880 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-557d84d488-5gmf7" podUID="088cf233-cbc9-45fb-b79a-cc5f1febf58d" containerName="manager" containerID="cri-o://ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8" gracePeriod=30 Jul 07 12:55:55.336265 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.336243 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:55:55.372906 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.372874 2574 generic.go:356] "Generic (PLEG): container finished" podID="088cf233-cbc9-45fb-b79a-cc5f1febf58d" containerID="ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8" exitCode=0 Jul 07 12:55:55.372906 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.372907 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-557d84d488-5gmf7" Jul 07 12:55:55.373116 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.372947 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-557d84d488-5gmf7" event={"ID":"088cf233-cbc9-45fb-b79a-cc5f1febf58d","Type":"ContainerDied","Data":"ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8"} Jul 07 12:55:55.373116 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.372995 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-557d84d488-5gmf7" event={"ID":"088cf233-cbc9-45fb-b79a-cc5f1febf58d","Type":"ContainerDied","Data":"f78537c11d011fd36584c867c9891d60ad1d202fd31213614dd9e8e8390c9c75"} Jul 07 12:55:55.373116 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.373022 2574 scope.go:117] "RemoveContainer" containerID="ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8" Jul 07 12:55:55.383631 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.383602 2574 scope.go:117] "RemoveContainer" containerID="ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8" Jul 07 12:55:55.384048 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:55:55.384026 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8\": container with ID starting with ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8 not found: ID does not exist" containerID="ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8" Jul 07 12:55:55.384150 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.384056 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8"} err="failed to get container status \"ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8\": rpc error: code = NotFound desc = could not find container \"ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8\": container with ID starting with ffd4b7b6e0ba312b6429fc7c07a20132fca46386c8268cbd1d4882f13ef8d0f8 not found: ID does not exist" Jul 07 12:55:55.455269 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.455240 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/088cf233-cbc9-45fb-b79a-cc5f1febf58d-webhook-cert\") pod \"088cf233-cbc9-45fb-b79a-cc5f1febf58d\" (UID: \"088cf233-cbc9-45fb-b79a-cc5f1febf58d\") " Jul 07 12:55:55.455431 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.455304 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgbz7\" (UniqueName: \"kubernetes.io/projected/088cf233-cbc9-45fb-b79a-cc5f1febf58d-kube-api-access-mgbz7\") pod \"088cf233-cbc9-45fb-b79a-cc5f1febf58d\" (UID: \"088cf233-cbc9-45fb-b79a-cc5f1febf58d\") " Jul 07 12:55:55.457282 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.457254 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/088cf233-cbc9-45fb-b79a-cc5f1febf58d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "088cf233-cbc9-45fb-b79a-cc5f1febf58d" (UID: "088cf233-cbc9-45fb-b79a-cc5f1febf58d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:55:55.457403 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.457289 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/088cf233-cbc9-45fb-b79a-cc5f1febf58d-kube-api-access-mgbz7" (OuterVolumeSpecName: "kube-api-access-mgbz7") pod "088cf233-cbc9-45fb-b79a-cc5f1febf58d" (UID: "088cf233-cbc9-45fb-b79a-cc5f1febf58d"). InnerVolumeSpecName "kube-api-access-mgbz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:55:55.556307 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.556228 2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/088cf233-cbc9-45fb-b79a-cc5f1febf58d-webhook-cert\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:55:55.556307 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.556260 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgbz7\" (UniqueName: \"kubernetes.io/projected/088cf233-cbc9-45fb-b79a-cc5f1febf58d-kube-api-access-mgbz7\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 12:55:55.698170 ip-10-0-135-212 kubenswrapper[2574]: E0707 12:55:55.698130 2574 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088cf233_cbc9_45fb_b79a_cc5f1febf58d.slice/crio-f78537c11d011fd36584c867c9891d60ad1d202fd31213614dd9e8e8390c9c75\": RecentStats: unable to find data in memory cache]" Jul 07 12:55:55.705294 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.705264 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-557d84d488-5gmf7"] Jul 07 12:55:55.708377 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:55.708350 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-557d84d488-5gmf7"] Jul 07 12:55:56.385233 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.385199 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:55:56.509299 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.509268 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-557d84d488-c6vgh"] Jul 07 12:55:56.509807 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.509792 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="088cf233-cbc9-45fb-b79a-cc5f1febf58d" containerName="manager" Jul 07 12:55:56.509870 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.509808 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="088cf233-cbc9-45fb-b79a-cc5f1febf58d" containerName="manager" Jul 07 12:55:56.509909 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.509899 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="088cf233-cbc9-45fb-b79a-cc5f1febf58d" containerName="manager" Jul 07 12:55:56.514336 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.514314 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-557d84d488-c6vgh" Jul 07 12:55:56.519102 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.519082 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 12:55:56.519234 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.519121 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zntz8\"" Jul 07 12:55:56.519234 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.519131 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 12:55:56.524763 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.524742 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-557d84d488-c6vgh"] Jul 07 12:55:56.564204 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.564178 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3dabe7cf-4ac3-4726-be83-0030845f15cc-webhook-cert\") pod \"maas-controller-557d84d488-c6vgh\" (UID: \"3dabe7cf-4ac3-4726-be83-0030845f15cc\") " pod="opendatahub/maas-controller-557d84d488-c6vgh" Jul 07 12:55:56.564320 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.564224 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcbd6\" (UniqueName: \"kubernetes.io/projected/3dabe7cf-4ac3-4726-be83-0030845f15cc-kube-api-access-qcbd6\") pod \"maas-controller-557d84d488-c6vgh\" (UID: \"3dabe7cf-4ac3-4726-be83-0030845f15cc\") " pod="opendatahub/maas-controller-557d84d488-c6vgh" Jul 07 12:55:56.665456 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.665395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcbd6\" (UniqueName: \"kubernetes.io/projected/3dabe7cf-4ac3-4726-be83-0030845f15cc-kube-api-access-qcbd6\") pod \"maas-controller-557d84d488-c6vgh\" (UID: \"3dabe7cf-4ac3-4726-be83-0030845f15cc\") " pod="opendatahub/maas-controller-557d84d488-c6vgh" Jul 07 12:55:56.665541 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.665493 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3dabe7cf-4ac3-4726-be83-0030845f15cc-webhook-cert\") pod \"maas-controller-557d84d488-c6vgh\" (UID: \"3dabe7cf-4ac3-4726-be83-0030845f15cc\") " pod="opendatahub/maas-controller-557d84d488-c6vgh" Jul 07 12:55:56.667819 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.667791 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3dabe7cf-4ac3-4726-be83-0030845f15cc-webhook-cert\") pod \"maas-controller-557d84d488-c6vgh\" (UID: \"3dabe7cf-4ac3-4726-be83-0030845f15cc\") " pod="opendatahub/maas-controller-557d84d488-c6vgh" Jul 07 12:55:56.675726 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.675704 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcbd6\" (UniqueName: \"kubernetes.io/projected/3dabe7cf-4ac3-4726-be83-0030845f15cc-kube-api-access-qcbd6\") pod \"maas-controller-557d84d488-c6vgh\" (UID: \"3dabe7cf-4ac3-4726-be83-0030845f15cc\") " pod="opendatahub/maas-controller-557d84d488-c6vgh" Jul 07 12:55:56.826492 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.826462 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-557d84d488-c6vgh" Jul 07 12:55:56.970523 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:56.970491 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-557d84d488-c6vgh"] Jul 07 12:55:56.971848 ip-10-0-135-212 kubenswrapper[2574]: W0707 12:55:56.971824 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dabe7cf_4ac3_4726_be83_0030845f15cc.slice/crio-4d96c84834c1520466f3615f737b416e57ec9c9e99b522a23a6b98060bd3f85d WatchSource:0}: Error finding container 4d96c84834c1520466f3615f737b416e57ec9c9e99b522a23a6b98060bd3f85d: Status 404 returned error can't find the container with id 4d96c84834c1520466f3615f737b416e57ec9c9e99b522a23a6b98060bd3f85d Jul 07 12:55:57.189713 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:57.189653 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="088cf233-cbc9-45fb-b79a-cc5f1febf58d" path="/var/lib/kubelet/pods/088cf233-cbc9-45fb-b79a-cc5f1febf58d/volumes" Jul 07 12:55:57.387675 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:57.387648 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-557d84d488-c6vgh" event={"ID":"3dabe7cf-4ac3-4726-be83-0030845f15cc","Type":"ContainerStarted","Data":"4d96c84834c1520466f3615f737b416e57ec9c9e99b522a23a6b98060bd3f85d"} Jul 07 12:55:58.394673 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:58.394630 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-557d84d488-c6vgh" event={"ID":"3dabe7cf-4ac3-4726-be83-0030845f15cc","Type":"ContainerStarted","Data":"7deefd619e44b7485450dd1c554da00fcd17752ae271bed561bcf484baaa911a"} Jul 07 12:55:58.395054 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:58.394703 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-557d84d488-c6vgh" Jul 07 12:55:58.413243 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:55:58.413200 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-557d84d488-c6vgh" podStartSLOduration=2.005794637 podStartE2EDuration="2.413184089s" totalImagesPullingTime="407.389452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:55:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:55:58.411649648 +0000 UTC m=+805.855692851" watchObservedRunningTime="2026-07-07 12:55:58.413184089 +0000 UTC m=+805.857227268" Jul 07 12:56:09.407827 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:09.407802 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-557d84d488-c6vgh" Jul 07 12:56:26.586912 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:26.586883 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:56:26.696698 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:26.696670 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:56:35.916675 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:35.916639 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:56:36.036896 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:36.036860 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:56:36.191327 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:36.191291 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:56:41.933309 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:41.933278 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:56:42.039376 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:42.039318 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:56:42.147875 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:42.147831 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:56:42.343294 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:42.343260 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:56:48.464061 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:48.464028 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:56:48.564117 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:56:48.564081 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:57:56.886511 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:57:56.886474 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 12:58:56.225636 ip-10-0-135-212 kubenswrapper[2574]: I0707 12:58:56.225542 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:00:35.261101 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:35.260990 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp"] Jul 07 13:00:35.261706 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:35.261478 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" podUID="cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f" containerName="manager" containerID="cri-o://c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215" gracePeriod=10 Jul 07 13:00:37.822853 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.822829 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 13:00:37.865870 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.865792 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-extensions-socket-volume\") pod \"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f\" (UID: \"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f\") " Jul 07 13:00:37.866025 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.865876 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqhjj\" (UniqueName: \"kubernetes.io/projected/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-kube-api-access-gqhjj\") pod \"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f\" (UID: \"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f\") " Jul 07 13:00:37.866210 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.866185 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f" (UID: "cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:00:37.867922 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.867896 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-kube-api-access-gqhjj" (OuterVolumeSpecName: "kube-api-access-gqhjj") pod "cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f" (UID: "cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f"). InnerVolumeSpecName "kube-api-access-gqhjj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:00:37.950251 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.950210 2574 generic.go:356] "Generic (PLEG): container finished" podID="cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f" containerID="c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215" exitCode=0 Jul 07 13:00:37.950251 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.950251 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" Jul 07 13:00:37.950469 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.950299 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" event={"ID":"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f","Type":"ContainerDied","Data":"c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215"} Jul 07 13:00:37.950469 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.950356 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp" event={"ID":"cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f","Type":"ContainerDied","Data":"27a8882b8f73395b7c17ce9686e5c0ac7f607d35f0a6712251a513d3532dbfdc"} Jul 07 13:00:37.950469 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.950388 2574 scope.go:117] "RemoveContainer" containerID="c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215" Jul 07 13:00:37.960731 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.960714 2574 scope.go:117] "RemoveContainer" containerID="c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215" Jul 07 13:00:37.961026 ip-10-0-135-212 kubenswrapper[2574]: E0707 13:00:37.961007 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215\": container with ID starting with c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215 not found: ID does not exist" containerID="c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215" Jul 07 13:00:37.961076 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.961034 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215"} err="failed to get container status \"c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215\": rpc error: code = NotFound desc = could not find container \"c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215\": container with ID starting with c214ee664a44ad661c41f67700dbf13f69295e8656d39458424582b8365d5215 not found: ID does not exist" Jul 07 13:00:37.966859 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.966840 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-extensions-socket-volume\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 13:00:37.966925 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.966860 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqhjj\" (UniqueName: \"kubernetes.io/projected/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f-kube-api-access-gqhjj\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 13:00:37.973184 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.973162 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp"] Jul 07 13:00:37.978312 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:37.978293 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jznp"] Jul 07 13:00:39.190330 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:00:39.190298 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f" path="/var/lib/kubelet/pods/cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f/volumes" Jul 07 13:01:36.482292 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.482257 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6"] Jul 07 13:01:36.482749 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.482735 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f" containerName="manager" Jul 07 13:01:36.482749 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.482750 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f" containerName="manager" Jul 07 13:01:36.482848 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.482831 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf4ab2d6-c2ba-4e24-8240-6bb69d398a2f" containerName="manager" Jul 07 13:01:36.486201 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.486186 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" Jul 07 13:01:36.491381 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.491363 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wht5w\"" Jul 07 13:01:36.501301 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.501275 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6"] Jul 07 13:01:36.639774 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.639737 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lc5r\" (UniqueName: \"kubernetes.io/projected/726e468c-87b7-4a01-8226-61516d234914-kube-api-access-9lc5r\") pod \"kuadrant-operator-controller-manager-86878c769f-2pph6\" (UID: \"726e468c-87b7-4a01-8226-61516d234914\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" Jul 07 13:01:36.639973 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.639820 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/726e468c-87b7-4a01-8226-61516d234914-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2pph6\" (UID: \"726e468c-87b7-4a01-8226-61516d234914\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" Jul 07 13:01:36.740892 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.740810 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lc5r\" (UniqueName: \"kubernetes.io/projected/726e468c-87b7-4a01-8226-61516d234914-kube-api-access-9lc5r\") pod \"kuadrant-operator-controller-manager-86878c769f-2pph6\" (UID: \"726e468c-87b7-4a01-8226-61516d234914\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" Jul 07 13:01:36.741045 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.740898 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/726e468c-87b7-4a01-8226-61516d234914-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2pph6\" (UID: \"726e468c-87b7-4a01-8226-61516d234914\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" Jul 07 13:01:36.741240 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.741225 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/726e468c-87b7-4a01-8226-61516d234914-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2pph6\" (UID: \"726e468c-87b7-4a01-8226-61516d234914\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" Jul 07 13:01:36.750757 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.750729 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lc5r\" (UniqueName: \"kubernetes.io/projected/726e468c-87b7-4a01-8226-61516d234914-kube-api-access-9lc5r\") pod \"kuadrant-operator-controller-manager-86878c769f-2pph6\" (UID: \"726e468c-87b7-4a01-8226-61516d234914\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" Jul 07 13:01:36.796644 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.796615 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" Jul 07 13:01:36.958639 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:36.958610 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6"] Jul 07 13:01:36.960264 ip-10-0-135-212 kubenswrapper[2574]: W0707 13:01:36.960231 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726e468c_87b7_4a01_8226_61516d234914.slice/crio-f4634446545cc412e3803ac54d47ad9d96ee2d3e921419364ba2b8bdd28d45f8 WatchSource:0}: Error finding container f4634446545cc412e3803ac54d47ad9d96ee2d3e921419364ba2b8bdd28d45f8: Status 404 returned error can't find the container with id f4634446545cc412e3803ac54d47ad9d96ee2d3e921419364ba2b8bdd28d45f8 Jul 07 13:01:37.276272 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:37.276188 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" event={"ID":"726e468c-87b7-4a01-8226-61516d234914","Type":"ContainerStarted","Data":"a05e9ed80e8bc60990ab0338c6f8980a85e498426c89e6b195ba34b3528fde0b"} Jul 07 13:01:37.276272 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:37.276241 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" event={"ID":"726e468c-87b7-4a01-8226-61516d234914","Type":"ContainerStarted","Data":"f4634446545cc412e3803ac54d47ad9d96ee2d3e921419364ba2b8bdd28d45f8"} Jul 07 13:01:37.276272 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:37.276262 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" Jul 07 13:01:37.301817 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:37.301761 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" podStartSLOduration=1.301744843 podStartE2EDuration="1.301744843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:01:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:01:37.299674397 +0000 UTC m=+1144.743717594" watchObservedRunningTime="2026-07-07 13:01:37.301744843 +0000 UTC m=+1144.745788022" Jul 07 13:01:48.284343 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:01:48.284309 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2pph6" Jul 07 13:02:38.904425 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:02:38.904388 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:02:39.071972 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:02:39.071930 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:04:22.789560 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:04:22.789529 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:04:24.630476 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:04:24.630438 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:05:16.849298 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:05:16.849207 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:05:55.235621 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:05:55.235583 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:18.561019 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:18.560977 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:18.705902 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:18.705864 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:24.051614 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:24.051583 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:24.116285 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:24.116245 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:24.276182 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:24.276135 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:24.346397 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:24.346300 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:25.458851 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:25.458820 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:25.549814 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:25.549769 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:25.642250 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:25.642212 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:26.044027 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:26.043986 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:26.150134 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:26.150085 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:28.292026 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:28.291994 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:30.515997 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:30.515959 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:30.725030 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:30.724991 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:30.802399 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:30.802319 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:30.903359 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:30.903324 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:32.227075 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:32.227039 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:36.705091 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:36.705030 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:36.807324 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:36.807286 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:36.926843 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:36.926805 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:37.025399 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:37.025315 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:38.014370 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:38.014331 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:38.186829 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:38.186794 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:38.336388 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:38.336302 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:38.430222 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:38.430192 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:38.806024 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:38.805977 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:38.906171 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:38.906138 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:39.940183 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:39.940139 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:43.152319 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:43.152282 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:43.506021 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:43.505981 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:49.287269 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:49.287234 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:49.373554 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:49.373518 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:55.615283 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:55.615197 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:55.719034 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:55.719004 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:55.830103 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:55.830048 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:56.857624 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:56.857524 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:56.984085 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:56.984052 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:57.105823 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:57.105784 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:57.205551 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:57.205488 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:57.304456 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:57.304428 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:57.860994 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:57.860960 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:07:57.966093 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:07:57.966053 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:00.558437 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:00.558404 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:01.385354 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:01.385320 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:01.611343 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:01.611307 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:01.714331 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:01.714293 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:01.828188 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:01.828152 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:03.679825 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:03.679793 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:07.617037 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:07.616988 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:07.716179 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:07.716139 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:08.113343 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:08.113299 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:08.306736 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:08.306680 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:08.451359 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:08.451320 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:09.840109 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:09.840079 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:15.807254 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:15.807211 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:15.938612 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:15.938563 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:16.035246 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:16.035208 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:16.134192 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:16.134100 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:16.539490 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:16.539451 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:42.533266 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:42.533226 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:42.826126 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:42.826028 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:42.929017 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:42.928980 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:48.748180 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:48.748146 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:48.857171 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:48.857141 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:08:48.951420 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:08:48.951384 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:04.927531 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:04.927496 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:10.686812 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:10.686773 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:10.793153 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:10.793101 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:10.889317 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:10.889281 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:21.468803 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:21.468766 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:22.295474 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:22.295444 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:28.515838 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:28.515741 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:30.243080 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:30.243048 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:31.599043 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:31.599003 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:37.837926 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:37.837877 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:37.940194 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:37.940156 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:38.532488 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:38.532455 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:44.959602 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:44.959564 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:45.139361 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:45.139327 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:45.242323 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:45.242231 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:45.603731 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:45.603615 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:45.696445 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:45.696410 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:48.580090 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:48.580055 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:48.676904 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:48.676866 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:48.784252 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:48.784215 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:48.886543 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:48.886453 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:49.447210 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:49.447173 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:49.550554 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:49.550519 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:53.120442 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:53.120407 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:53.843623 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:53.843578 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:53.957299 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:53.957267 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:09:54.520775 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:09:54.520735 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:00.595398 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:00.595360 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:00.660128 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:00.660096 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:01.133100 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:01.133055 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:01.585254 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:01.585221 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:01.857263 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:01.857174 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:07.829110 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:07.829060 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:07.929435 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:07.929393 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:08.114545 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:08.114455 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:08.516541 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:08.516499 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:08.616786 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:08.616745 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:08.740755 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:08.740720 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:14.586468 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:14.586435 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:14.691523 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:14.691479 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:14.788476 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:14.788442 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:14.915944 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:14.915841 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:21.560496 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:21.560447 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:21.659255 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:21.659221 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:28.568150 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:28.568110 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:28.913552 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:28.913465 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:29.168211 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:29.168105 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:31.696174 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:31.696133 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:31.786110 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:31.786073 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:31.928467 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:31.928418 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:32.063828 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:32.063734 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:32.399905 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:32.399814 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:32.501028 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:32.500983 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:36.217857 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:36.217823 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:40.886519 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:40.886469 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:58.537224 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:58.537188 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:10:58.947609 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:10:58.947579 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:11:04.657735 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:11:04.657672 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:11:09.741229 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:11:09.741190 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:11:17.734299 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:11:17.734258 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:11:17.835722 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:11:17.835674 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:11:21.128987 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:11:21.128955 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:11:46.403546 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:11:46.403508 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:11:47.855340 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:11:47.855294 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:11:53.665182 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:11:53.665141 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:00.390715 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:00.390659 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:00.482434 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:00.482399 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:00.766633 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:00.766594 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:02.052184 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:02.052141 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:02.165231 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:02.165196 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:02.392141 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:02.392046 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:03.024592 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:03.024544 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:03.127588 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:03.127545 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:03.223975 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:03.223930 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:04.693813 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:04.693765 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:10.913022 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:10.912981 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:10.990071 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:10.990032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:11.603172 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:11.603139 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:18.602811 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:18.602769 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:18.750830 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:18.750792 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:19.031162 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:19.031129 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:29.456139 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:29.456055 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:29.825703 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:29.825603 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:50.768243 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:50.768199 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:57.210832 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:57.210786 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:57.312273 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:57.312218 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:57.605523 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:57.605440 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:57.969716 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:57.969660 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:12:58.087042 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:12:58.087001 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:01.428322 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:01.428292 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:01.528261 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:01.528219 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:01.715588 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:01.715550 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:01.937939 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:01.937898 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:02.192107 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:02.192073 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:02.292743 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:02.292678 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:05.972896 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:05.972858 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:06.334478 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:06.334380 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:06.663586 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:06.663500 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:06.776159 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:06.776125 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:09.406279 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:09.406222 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:09.511421 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:09.511277 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:09.587164 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:09.587116 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:09.705278 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:09.705245 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:10.083578 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:10.083495 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:10.184308 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:10.184263 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:13.932516 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:13.932482 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:23.231056 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:23.231018 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:23.330936 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:23.330904 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:31.782442 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:31.782403 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:31.884528 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:31.884489 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:32.715351 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:32.715306 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:33.150841 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:33.150737 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:13:58.840926 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:13:58.840889 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:00.101394 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:14:00.101361 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:17.752938 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:14:17.752899 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:18.762795 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:14:18.762763 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:18.917471 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:14:18.917429 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:19.450971 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:14:19.450923 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:19.628808 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:14:19.628770 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:19.733973 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:14:19.733882 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:19.894823 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:14:19.894767 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:14:24.538522 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:14:24.538462 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:00.146262 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:00.146200 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6"] Jul 07 13:15:00.150151 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:00.150127 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" Jul 07 13:15:00.157966 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:00.157918 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6"] Jul 07 13:15:00.251428 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:00.251366 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt74v\" (UniqueName: \"kubernetes.io/projected/a18a2dfa-fc4a-403d-b086-1d1d1675eea6-kube-api-access-jt74v\") pod \"maas-api-key-cleanup-29723835-gdmm6\" (UID: \"a18a2dfa-fc4a-403d-b086-1d1d1675eea6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" Jul 07 13:15:00.352534 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:00.352476 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt74v\" (UniqueName: \"kubernetes.io/projected/a18a2dfa-fc4a-403d-b086-1d1d1675eea6-kube-api-access-jt74v\") pod \"maas-api-key-cleanup-29723835-gdmm6\" (UID: \"a18a2dfa-fc4a-403d-b086-1d1d1675eea6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" Jul 07 13:15:00.360803 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:00.360776 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt74v\" (UniqueName: \"kubernetes.io/projected/a18a2dfa-fc4a-403d-b086-1d1d1675eea6-kube-api-access-jt74v\") pod \"maas-api-key-cleanup-29723835-gdmm6\" (UID: \"a18a2dfa-fc4a-403d-b086-1d1d1675eea6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" Jul 07 13:15:00.462377 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:00.462344 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" Jul 07 13:15:00.606253 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:00.606223 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6"] Jul 07 13:15:00.608357 ip-10-0-135-212 kubenswrapper[2574]: W0707 13:15:00.608327 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18a2dfa_fc4a_403d_b086_1d1d1675eea6.slice/crio-7cdd9f1f05c906b4ce6e0895b50a33818fc5dcd9403754d6bd3e1dedf446744f WatchSource:0}: Error finding container 7cdd9f1f05c906b4ce6e0895b50a33818fc5dcd9403754d6bd3e1dedf446744f: Status 404 returned error can't find the container with id 7cdd9f1f05c906b4ce6e0895b50a33818fc5dcd9403754d6bd3e1dedf446744f Jul 07 13:15:00.610334 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:00.610316 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:15:00.801094 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:00.801000 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" event={"ID":"a18a2dfa-fc4a-403d-b086-1d1d1675eea6","Type":"ContainerStarted","Data":"7cdd9f1f05c906b4ce6e0895b50a33818fc5dcd9403754d6bd3e1dedf446744f"} Jul 07 13:15:01.569217 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:01.569139 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:03.681761 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:03.681724 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:03.823667 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:03.823631 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" event={"ID":"a18a2dfa-fc4a-403d-b086-1d1d1675eea6","Type":"ContainerStarted","Data":"0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75"} Jul 07 13:15:03.841416 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:03.841366 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" podStartSLOduration=1.9015945140000001 podStartE2EDuration="3.841349824s" totalImagesPullingTime="1.93975531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:03.839519087 +0000 UTC m=+1951.283562288" watchObservedRunningTime="2026-07-07 13:15:03.841349824 +0000 UTC m=+1951.285393009" Jul 07 13:15:08.505986 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:08.505946 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:08.607398 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:08.607342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:08.812181 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:08.812084 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:09.456203 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:09.456169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:09.834838 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:09.834727 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:09.937889 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:09.937852 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:13.020310 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:13.020264 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:13.095251 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:13.095212 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:13.169277 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:13.169231 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:13.270417 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:13.270324 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:13.639206 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:13.639119 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:13.737143 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:13.737107 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:17.532414 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:17.532376 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:18.008079 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:18.008037 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:18.320839 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:18.320742 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:18.587188 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:18.587090 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:21.042972 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:21.042932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:21.144728 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:21.144676 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:21.246232 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:21.246194 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:21.416012 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:21.415928 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:21.953509 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:21.953473 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:22.022232 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:22.022189 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:25.571409 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:25.571309 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:33.962108 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:33.962077 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:34.327760 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:34.327665 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:34.471273 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:34.471238 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:34.572226 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:34.572169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:34.738132 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:34.738099 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:40.243055 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:40.243009 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:40.342610 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:40.342574 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:40.518805 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:40.518675 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:40.899068 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:40.898975 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:41.318716 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:41.318652 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:41.419299 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:41.419264 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:41.521814 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:41.521781 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:41.761387 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:41.761353 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:47.450955 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:47.450917 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:47.522631 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:47.522593 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:47.711541 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:47.711448 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:48.307947 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:48.307905 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:48.710182 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:48.710146 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:49.014224 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:49.014122 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:51.308995 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:51.308937 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:51.467589 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:51.467551 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:51.544449 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:51.544408 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:51.643099 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:51.642996 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:52.058999 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:52.058965 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:52.159946 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:52.159915 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:55.767173 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:55.767064 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:56.105540 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:56.105443 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:56.436756 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:56.436646 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:56.622175 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:56.622140 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:59.271020 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:59.270988 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:59.383221 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:59.383185 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:59.509789 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:59.509743 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:59.888756 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:59.888717 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:15:59.986241 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:15:59.986205 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:00.012606 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:00.012568 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6"] Jul 07 13:16:00.012851 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:00.012828 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" podUID="a18a2dfa-fc4a-403d-b086-1d1d1675eea6" containerName="cleanup" containerID="cri-o://0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75" gracePeriod=30 Jul 07 13:16:00.075205 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:00.075169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:03.782657 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:03.782611 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:12.941033 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:12.940992 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:13.016212 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:13.016177 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:18.097928 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:18.097884 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:18.467339 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:18.467301 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:18.568110 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:18.568067 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:20.747293 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:20.747244 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:20.847250 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:20.847215 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:30.201206 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.201179 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" Jul 07 13:16:30.291652 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.291567 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt74v\" (UniqueName: \"kubernetes.io/projected/a18a2dfa-fc4a-403d-b086-1d1d1675eea6-kube-api-access-jt74v\") pod \"a18a2dfa-fc4a-403d-b086-1d1d1675eea6\" (UID: \"a18a2dfa-fc4a-403d-b086-1d1d1675eea6\") " Jul 07 13:16:30.293726 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.293700 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18a2dfa-fc4a-403d-b086-1d1d1675eea6-kube-api-access-jt74v" (OuterVolumeSpecName: "kube-api-access-jt74v") pod "a18a2dfa-fc4a-403d-b086-1d1d1675eea6" (UID: "a18a2dfa-fc4a-403d-b086-1d1d1675eea6"). InnerVolumeSpecName "kube-api-access-jt74v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:16:30.316092 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.316056 2574 generic.go:356] "Generic (PLEG): container finished" podID="a18a2dfa-fc4a-403d-b086-1d1d1675eea6" containerID="0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75" exitCode=137 Jul 07 13:16:30.316092 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.316100 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" Jul 07 13:16:30.316324 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.316141 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" event={"ID":"a18a2dfa-fc4a-403d-b086-1d1d1675eea6","Type":"ContainerDied","Data":"0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75"} Jul 07 13:16:30.316324 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.316201 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6" event={"ID":"a18a2dfa-fc4a-403d-b086-1d1d1675eea6","Type":"ContainerDied","Data":"7cdd9f1f05c906b4ce6e0895b50a33818fc5dcd9403754d6bd3e1dedf446744f"} Jul 07 13:16:30.316324 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.316227 2574 scope.go:117] "RemoveContainer" containerID="0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75" Jul 07 13:16:30.326578 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.326558 2574 scope.go:117] "RemoveContainer" containerID="0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75" Jul 07 13:16:30.326986 ip-10-0-135-212 kubenswrapper[2574]: E0707 13:16:30.326967 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75\": container with ID starting with 0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75 not found: ID does not exist" containerID="0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75" Jul 07 13:16:30.327040 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.326995 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75"} err="failed to get container status \"0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75\": rpc error: code = NotFound desc = could not find container \"0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75\": container with ID starting with 0bd5dee61d63159ee25ff5215bf3a3b0d7794c9ee8d4c3b91d67265d3e1b6d75 not found: ID does not exist" Jul 07 13:16:30.339742 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.339709 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6"] Jul 07 13:16:30.343450 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.343426 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723835-gdmm6"] Jul 07 13:16:30.392903 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:30.392877 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt74v\" (UniqueName: \"kubernetes.io/projected/a18a2dfa-fc4a-403d-b086-1d1d1675eea6-kube-api-access-jt74v\") on node \"ip-10-0-135-212.ec2.internal\" DevicePath \"\"" Jul 07 13:16:31.195359 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:31.195324 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a18a2dfa-fc4a-403d-b086-1d1d1675eea6" path="/var/lib/kubelet/pods/a18a2dfa-fc4a-403d-b086-1d1d1675eea6/volumes" Jul 07 13:16:34.652893 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:34.652854 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:16:42.359905 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:16:42.359867 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:37.142011 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:18:37.141932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:37.229015 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:18:37.228980 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:37.311262 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:18:37.311218 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:38.195730 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:18:38.195670 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:18:41.001600 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:18:41.001550 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:00.362074 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:19:00.362041 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:16.671274 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:19:16.671236 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:31.633186 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:19:31.633152 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:31.736826 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:19:31.736792 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:54.683284 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:19:54.683186 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:19:55.663199 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:19:55.663061 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:03.077310 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:20:03.077269 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:20:27.625987 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:20:27.625948 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:13.684056 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:13.684017 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:13.783735 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:13.783683 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:17.492078 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:17.492039 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:18.626394 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:18.626360 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:19.279823 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:19.279783 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:19.379361 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:19.379324 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:19.767703 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:19.767660 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:20.167224 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:20.167138 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:20.322281 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:20.322244 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:20.414087 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:20.414041 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:20.631426 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:20.631390 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:21.552506 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:21.552469 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:21.653385 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:21.653322 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:21:25.226248 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:21:25.226166 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:02.863428 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:02.863391 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:03.162151 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:03.162060 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:03.552919 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:03.552842 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:03.992297 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:03.992263 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:04.093804 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:04.093762 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:10.233856 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:10.233807 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:10.298397 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:10.298335 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:10.529337 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:10.529222 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:16.027421 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:16.027378 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xkgpz_d68a452b-a3eb-41a4-ad62-13792602001d/manager/0.log" Jul 07 13:22:16.153193 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:16.153161 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mr8fq_51726836-56b5-40bf-a16e-766b86b9d750/manager/0.log" Jul 07 13:22:16.280625 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:16.280536 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-557d84d488-c6vgh_3dabe7cf-4ac3-4726-be83-0030845f15cc/manager/0.log" Jul 07 13:22:16.410366 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:16.410336 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bzhg7_5262fe54-c1bd-4ca2-bb12-46b8ba16254e/server/0.log" Jul 07 13:22:16.542813 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:16.542719 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ffz26_a0624566-5357-4498-bc6b-92c36bb05ded/manager/0.log" Jul 07 13:22:16.930438 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:16.930355 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-x4sss_6662221e-c29d-40b4-a12b-b6724932a97b/copy-manifests/0.log" Jul 07 13:22:16.947658 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:16.947632 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-x4sss_6662221e-c29d-40b4-a12b-b6724932a97b/manager/0.log" Jul 07 13:22:18.494736 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:18.494707 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bwhpj_ae483d2d-ad2b-4afd-bc5e-e4c7726a4819/manager/0.log" Jul 07 13:22:18.620566 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:18.620538 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nd6gr_005a2d2b-5a6e-42d5-9e0d-f7c76a89c436/manager/0.log" Jul 07 13:22:18.868295 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:18.868211 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-qq7mv_4b2d4000-e605-4dda-9574-a7f5879e6530/registry-server/0.log" Jul 07 13:22:18.997124 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:18.997094 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2pph6_726e468c-87b7-4a01-8226-61516d234914/manager/0.log" Jul 07 13:22:19.253928 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:19.253881 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-7dqxh_9b049194-41bb-400f-ba08-9d205c54ea82/manager/0.log" Jul 07 13:22:20.033855 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:20.033830 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74684786d9-k2xkz_66201680-5f59-4d9f-a032-f81913957842/kube-auth-proxy/0.log" Jul 07 13:22:20.159930 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:20.159904 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qvw7t_3e9c599b-ee72-4420-b164-d46c0e5e1049/istio-proxy/1.log" Jul 07 13:22:20.286479 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:20.286397 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jw8hs_d35dfec3-b630-4978-95a2-7276835fb5d8/payload-pre-processing/0.log" Jul 07 13:22:20.421033 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:20.421002 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7dmtd_e29d948e-020c-41f3-a03e-ddd60aea93cc/payload-processing/0.log" Jul 07 13:22:21.031464 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:21.031437 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-hn8jn_e7b2a822-1e46-49ff-8917-71654365e360/main/0.log" Jul 07 13:22:21.150925 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:21.150898 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jx4q2_60121699-e818-4fa6-8277-ce2ada9a7acf/main/0.log" Jul 07 13:22:25.582435 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:25.582401 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:25.649999 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:25.649968 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:29.239044 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:29.239006 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" limit=1024 Jul 07 13:22:29.296463 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:29.296434 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cbl98_5e3e1d99-1fd1-4f01-980b-146f14cb0b43/global-pull-secret-syncer/0.log" Jul 07 13:22:29.426555 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:29.426523 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2dtxn_cc610f8c-e866-4ff6-8a27-172ddb224a0c/konnectivity-agent/0.log" Jul 07 13:22:29.645786 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:29.645684 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-212.ec2.internal_37ae1767e02e91e0c33f5647bfe3b14c/haproxy/0.log" Jul 07 13:22:34.112993 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:34.112893 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bwhpj_ae483d2d-ad2b-4afd-bc5e-e4c7726a4819/manager/0.log" Jul 07 13:22:34.141510 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:34.141471 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nd6gr_005a2d2b-5a6e-42d5-9e0d-f7c76a89c436/manager/0.log" Jul 07 13:22:34.212079 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:34.212051 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-qq7mv_4b2d4000-e605-4dda-9574-a7f5879e6530/registry-server/0.log" Jul 07 13:22:34.348332 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:34.348290 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2pph6_726e468c-87b7-4a01-8226-61516d234914/manager/0.log" Jul 07 13:22:34.460510 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:34.460474 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-7dqxh_9b049194-41bb-400f-ba08-9d205c54ea82/manager/0.log" Jul 07 13:22:36.061912 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.061864 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_634c42b9-63dd-404e-8e83-a330fb6d5b49/alertmanager/0.log" Jul 07 13:22:36.086910 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.086831 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_634c42b9-63dd-404e-8e83-a330fb6d5b49/config-reloader/0.log" Jul 07 13:22:36.110621 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.110583 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_634c42b9-63dd-404e-8e83-a330fb6d5b49/kube-rbac-proxy-web/0.log" Jul 07 13:22:36.140957 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.140930 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_634c42b9-63dd-404e-8e83-a330fb6d5b49/kube-rbac-proxy/0.log" Jul 07 13:22:36.165141 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.165113 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_634c42b9-63dd-404e-8e83-a330fb6d5b49/kube-rbac-proxy-metric/0.log" Jul 07 13:22:36.189327 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.189306 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_634c42b9-63dd-404e-8e83-a330fb6d5b49/prom-label-proxy/0.log" Jul 07 13:22:36.211602 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.211575 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_634c42b9-63dd-404e-8e83-a330fb6d5b49/init-config-reloader/0.log" Jul 07 13:22:36.299733 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.299703 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-gr8cr_ce62cb92-033c-4abc-a117-b3896d689f23/kube-state-metrics/0.log" Jul 07 13:22:36.319963 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.319937 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-gr8cr_ce62cb92-033c-4abc-a117-b3896d689f23/kube-rbac-proxy-main/0.log" Jul 07 13:22:36.345983 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.345897 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-gr8cr_ce62cb92-033c-4abc-a117-b3896d689f23/kube-rbac-proxy-self/0.log" Jul 07 13:22:36.379675 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.379639 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-559b4857cc-ktlxm_f06fe04d-b6bc-4540-ad4a-974aaa983a22/metrics-server/0.log" Jul 07 13:22:36.413006 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.412979 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-96x9t_6c4f650c-ff41-433e-abdc-fc6503aee125/monitoring-plugin/0.log" Jul 07 13:22:36.668128 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.668047 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z6fdd_6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e/node-exporter/0.log" Jul 07 13:22:36.702095 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.702066 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z6fdd_6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e/kube-rbac-proxy/0.log" Jul 07 13:22:36.731946 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.731918 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z6fdd_6e7a3f2a-55f6-4e2d-be5e-c4094adfed8e/init-textfile/0.log" Jul 07 13:22:36.770326 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.770298 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-wnn8b_3644ab2c-4c37-4b02-b77c-bee617c5ed46/kube-rbac-proxy-main/0.log" Jul 07 13:22:36.797954 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.797929 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-wnn8b_3644ab2c-4c37-4b02-b77c-bee617c5ed46/kube-rbac-proxy-self/0.log" Jul 07 13:22:36.823572 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.823545 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-wnn8b_3644ab2c-4c37-4b02-b77c-bee617c5ed46/openshift-state-metrics/0.log" Jul 07 13:22:36.865219 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.865189 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d968574c-da0c-4d18-adc4-9d76d99a8035/prometheus/0.log" Jul 07 13:22:36.889359 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.889336 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d968574c-da0c-4d18-adc4-9d76d99a8035/config-reloader/0.log" Jul 07 13:22:36.914391 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.914368 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d968574c-da0c-4d18-adc4-9d76d99a8035/thanos-sidecar/0.log" Jul 07 13:22:36.938601 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.938531 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d968574c-da0c-4d18-adc4-9d76d99a8035/kube-rbac-proxy-web/0.log" Jul 07 13:22:36.966952 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.966926 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d968574c-da0c-4d18-adc4-9d76d99a8035/kube-rbac-proxy/0.log" Jul 07 13:22:36.995001 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:36.994979 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d968574c-da0c-4d18-adc4-9d76d99a8035/kube-rbac-proxy-thanos/0.log" Jul 07 13:22:37.020269 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.020238 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d968574c-da0c-4d18-adc4-9d76d99a8035/init-config-reloader/0.log" Jul 07 13:22:37.051510 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.051486 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-7qjv2_ffacd4a8-e132-44cc-9bf9-a280a39a0daf/prometheus-operator/0.log" Jul 07 13:22:37.072193 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.072171 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-7qjv2_ffacd4a8-e132-44cc-9bf9-a280a39a0daf/kube-rbac-proxy/0.log" Jul 07 13:22:37.143391 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.143363 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d5b88bb68-n8z2v_207bfd0e-99c9-4342-bf59-52261564614e/telemeter-client/0.log" Jul 07 13:22:37.169456 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.169431 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d5b88bb68-n8z2v_207bfd0e-99c9-4342-bf59-52261564614e/reload/0.log" Jul 07 13:22:37.193101 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.193075 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d5b88bb68-n8z2v_207bfd0e-99c9-4342-bf59-52261564614e/kube-rbac-proxy/0.log" Jul 07 13:22:37.230174 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.230146 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6686974f76-rmcwm_b6591db7-8228-485c-aa88-13ebba70998c/thanos-query/0.log" Jul 07 13:22:37.252820 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.252795 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6686974f76-rmcwm_b6591db7-8228-485c-aa88-13ebba70998c/kube-rbac-proxy-web/0.log" Jul 07 13:22:37.277881 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.277857 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6686974f76-rmcwm_b6591db7-8228-485c-aa88-13ebba70998c/kube-rbac-proxy/0.log" Jul 07 13:22:37.301197 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.301173 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6686974f76-rmcwm_b6591db7-8228-485c-aa88-13ebba70998c/prom-label-proxy/0.log" Jul 07 13:22:37.323500 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.323477 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6686974f76-rmcwm_b6591db7-8228-485c-aa88-13ebba70998c/kube-rbac-proxy-rules/0.log" Jul 07 13:22:37.349795 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:37.349768 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6686974f76-rmcwm_b6591db7-8228-485c-aa88-13ebba70998c/kube-rbac-proxy-metrics/0.log" Jul 07 13:22:39.229016 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.228978 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc"] Jul 07 13:22:39.229471 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.229457 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18a2dfa-fc4a-403d-b086-1d1d1675eea6" containerName="cleanup" Jul 07 13:22:39.229516 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.229473 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18a2dfa-fc4a-403d-b086-1d1d1675eea6" containerName="cleanup" Jul 07 13:22:39.229578 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.229566 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18a2dfa-fc4a-403d-b086-1d1d1675eea6" containerName="cleanup" Jul 07 13:22:39.232514 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.232499 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.235337 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.235316 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gwgv6\"/\"openshift-service-ca.crt\"" Jul 07 13:22:39.235417 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.235316 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gwgv6\"/\"kube-root-ca.crt\"" Jul 07 13:22:39.236708 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.236677 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gwgv6\"/\"default-dockercfg-65jtm\"" Jul 07 13:22:39.243033 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.243007 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc"] Jul 07 13:22:39.324076 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.324035 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-proc\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.324273 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.324094 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-lib-modules\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.324273 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.324147 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-podres\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.324273 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.324234 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-sys\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.324435 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.324308 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnkbb\" (UniqueName: \"kubernetes.io/projected/b20fd84b-b279-466a-a6c7-22daa99c260a-kube-api-access-gnkbb\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.425530 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.425497 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-sys\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.425743 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.425574 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnkbb\" (UniqueName: \"kubernetes.io/projected/b20fd84b-b279-466a-a6c7-22daa99c260a-kube-api-access-gnkbb\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.425743 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.425629 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-sys\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.425743 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.425651 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-proc\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.425743 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.425734 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-proc\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.425982 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.425801 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-lib-modules\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.425982 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.425865 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-podres\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.425982 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.425923 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-lib-modules\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.426106 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.425985 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b20fd84b-b279-466a-a6c7-22daa99c260a-podres\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.435721 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.435661 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnkbb\" (UniqueName: \"kubernetes.io/projected/b20fd84b-b279-466a-a6c7-22daa99c260a-kube-api-access-gnkbb\") pod \"perf-node-gather-daemonset-2pxrc\" (UID: \"b20fd84b-b279-466a-a6c7-22daa99c260a\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.544200 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.544104 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:39.688403 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.688360 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc"] Jul 07 13:22:39.691739 ip-10-0-135-212 kubenswrapper[2574]: W0707 13:22:39.691683 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb20fd84b_b279_466a_a6c7_22daa99c260a.slice/crio-da80a881f7b1836c7417c45091ece6bf85c20a409a94e8e5366a6ec8b987e8b3 WatchSource:0}: Error finding container da80a881f7b1836c7417c45091ece6bf85c20a409a94e8e5366a6ec8b987e8b3: Status 404 returned error can't find the container with id da80a881f7b1836c7417c45091ece6bf85c20a409a94e8e5366a6ec8b987e8b3 Jul 07 13:22:39.854759 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:39.854728 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-dc9556d4b-8k5rm_440b7cb4-d1d7-416a-9a42-fe6960cb679b/console/0.log" Jul 07 13:22:40.379315 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:40.379281 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" event={"ID":"b20fd84b-b279-466a-a6c7-22daa99c260a","Type":"ContainerStarted","Data":"90cd728a5755d77ddc963ce1e46832346e24e9eeecf8ac00b32dc0dd9d33a390"} Jul 07 13:22:40.379766 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:40.379338 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" event={"ID":"b20fd84b-b279-466a-a6c7-22daa99c260a","Type":"ContainerStarted","Data":"da80a881f7b1836c7417c45091ece6bf85c20a409a94e8e5366a6ec8b987e8b3"} Jul 07 13:22:40.379766 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:40.379466 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:40.396442 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:40.396392 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" podStartSLOduration=1.396373823 podStartE2EDuration="1.396373823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:22:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:40.394685049 +0000 UTC m=+2407.838728229" watchObservedRunningTime="2026-07-07 13:22:40.396373823 +0000 UTC m=+2407.840417001" Jul 07 13:22:41.780878 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:41.780851 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vfdz6_16e2ac83-43c2-439f-994a-943d545f9aba/dns/0.log" Jul 07 13:22:41.803107 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:41.803066 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vfdz6_16e2ac83-43c2-439f-994a-943d545f9aba/kube-rbac-proxy/0.log" Jul 07 13:22:41.826573 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:41.826548 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6f6bh_ae03df12-73df-481b-b330-c624c016e8cb/dns-node-resolver/0.log" Jul 07 13:22:42.467024 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:42.466992 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qrr6k_3a67391a-c8bc-4eaf-a00a-c728f059574c/node-ca/0.log" Jul 07 13:22:43.724267 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:43.724237 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74684786d9-k2xkz_66201680-5f59-4d9f-a032-f81913957842/kube-auth-proxy/0.log" Jul 07 13:22:43.758455 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:43.758425 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qvw7t_3e9c599b-ee72-4420-b164-d46c0e5e1049/istio-proxy/0.log" Jul 07 13:22:43.795370 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:43.795340 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qvw7t_3e9c599b-ee72-4420-b164-d46c0e5e1049/istio-proxy/1.log" Jul 07 13:22:43.825387 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:43.825353 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jw8hs_d35dfec3-b630-4978-95a2-7276835fb5d8/payload-pre-processing/0.log" Jul 07 13:22:43.872157 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:43.872124 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7dmtd_e29d948e-020c-41f3-a03e-ddd60aea93cc/payload-processing/0.log" Jul 07 13:22:44.622337 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:44.622306 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qxgh8_70dd9ad2-6e2e-492e-801b-6b9965957746/serve-healthcheck-canary/0.log" Jul 07 13:22:45.341166 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:45.341135 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xvjb2_f5da8ea0-29bd-4abf-803d-1f76999d78a3/kube-rbac-proxy/0.log" Jul 07 13:22:45.364880 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:45.364855 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xvjb2_f5da8ea0-29bd-4abf-803d-1f76999d78a3/exporter/0.log" Jul 07 13:22:45.389362 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:45.389337 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xvjb2_f5da8ea0-29bd-4abf-803d-1f76999d78a3/extractor/0.log" Jul 07 13:22:46.395760 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:46.395729 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-2pxrc" Jul 07 13:22:47.634725 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:47.634622 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xkgpz_d68a452b-a3eb-41a4-ad62-13792602001d/manager/0.log" Jul 07 13:22:47.766449 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:47.766418 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mr8fq_51726836-56b5-40bf-a16e-766b86b9d750/manager/0.log" Jul 07 13:22:47.876896 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:47.876864 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-557d84d488-c6vgh_3dabe7cf-4ac3-4726-be83-0030845f15cc/manager/0.log" Jul 07 13:22:47.897146 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:47.897074 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bzhg7_5262fe54-c1bd-4ca2-bb12-46b8ba16254e/server/0.log" Jul 07 13:22:47.960472 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:47.960437 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ffz26_a0624566-5357-4498-bc6b-92c36bb05ded/manager/0.log" Jul 07 13:22:48.142122 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:48.142091 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-x4sss_6662221e-c29d-40b4-a12b-b6724932a97b/manager/0.log" Jul 07 13:22:48.161876 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:48.161801 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-x4sss_6662221e-c29d-40b4-a12b-b6724932a97b/copy-manifests/0.log" Jul 07 13:22:49.905622 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:49.905596 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74684786d9-k2xkz_66201680-5f59-4d9f-a032-f81913957842/kube-auth-proxy/0.log" Jul 07 13:22:49.927753 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:49.927714 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qvw7t_3e9c599b-ee72-4420-b164-d46c0e5e1049/istio-proxy/0.log" Jul 07 13:22:49.956497 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:49.956462 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qvw7t_3e9c599b-ee72-4420-b164-d46c0e5e1049/istio-proxy/1.log" Jul 07 13:22:49.978014 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:49.977985 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jw8hs_d35dfec3-b630-4978-95a2-7276835fb5d8/payload-pre-processing/0.log" Jul 07 13:22:50.016848 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:50.016819 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7dmtd_e29d948e-020c-41f3-a03e-ddd60aea93cc/payload-processing/0.log" Jul 07 13:22:51.537337 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:51.537308 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74684786d9-k2xkz_66201680-5f59-4d9f-a032-f81913957842/kube-auth-proxy/0.log" Jul 07 13:22:51.565553 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:51.565521 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qvw7t_3e9c599b-ee72-4420-b164-d46c0e5e1049/istio-proxy/0.log" Jul 07 13:22:51.590521 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:51.590490 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qvw7t_3e9c599b-ee72-4420-b164-d46c0e5e1049/istio-proxy/1.log" Jul 07 13:22:51.614923 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:51.614890 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jw8hs_d35dfec3-b630-4978-95a2-7276835fb5d8/payload-pre-processing/0.log" Jul 07 13:22:51.669527 ip-10-0-135-212 kubenswrapper[2574]: I0707 13:22:51.669498 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-7dmtd_e29d948e-020c-41f3-a03e-ddd60aea93cc/payload-processing/0.log"